Dynamic Spectrum Access in Time-Varying Environment: Distributed Learning Beyond Expectation Optimization

• Published in: IEEE transactions on communications Vol. 65; no. 12; pp. 5305 - 5318
• Main Authors: Yuhua Xu, Jinlong Wang, Qihui Wu, Jianchao Zheng, Liang Shen, Anpalagan, Alagan
• Format: Journal Article
• Language: English
• Published: New York IEEE 01.12.2017; The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
• Subjects: Algorithms; Computer simulation; dense networks; Dynamic spectrum access; Economic models; effective capacity; Game theory; Games; Heuristic algorithms; Machine learning; Measurement; multi-agent learning; Multiagent systems; Optimization; potential game; Quality of service; Statistical analysis; statistical QoS; Variations; Wireless networks
• ISSN: 0090-6778; 1558-0857
• DOI: 10.1109/TCOMM.2017.2734768

This paper investigates the problem of dynamic spectrum access for canonical wireless networks, in which the channel states are time-varying. In the most existing work, the commonly used optimization objective is to maximize the expectation of a certain metric (e.g., throughput or achievable rate). However, it is realized that expectation alone is not enough since some applications are sensitive to fluctuations. Effective capacity is a promising metric for time-varying service process since it characterizes the packet delay violating probability (regarded as an important statistical quality-of-service index), by taking into account not only the expectation but also other high-order statistic. Therefore, we formulate the interactions among the users in the time-varying environment as a non-cooperative game, in which the utility function is defined as the achieved effective capacity. We prove that it is an ordinal potential game which has at least one pure strategy Nash equilibrium. Based on an approximated utility function, we propose a multi-agent learning algorithm which is proved to achieve stable solutions with dynamic and incomplete information constraints. The convergence of the proposed learning algorithm is verified by simulation results. Also, it is shown that the proposed multi-agent learning algorithm achieves satisfactory performance.