Dynamic Multichannel Access via Multi-Agent Reinforcement Learning: Throughput and Fairness Guarantees

• Published in: IEEE transactions on wireless communications Vol. 21; no. 6; pp. 3994 - 4008
• Main Authors: Sohaib, Muhammad, Jeong, Jongjin, Jeon, Sang-Woon
• Format: Journal Article
• Language: English
• Published: New York IEEE 01.06.2022; The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
• Subjects: Algorithms; Computer architecture; deep learning; Distance learning; fairness; Machine learning; Measurement; Multiagent systems; Policies; Quality of service; Random access; Real-time systems; Reinforcement learning; resource allocation; Resource management; Throughput; Wireless communication
• ISSN: 1536-1276; 1558-2248
• DOI: 10.1109/TWC.2021.3126112

We consider a multichannel random access system in which each user accesses a single channel at each time slot to communicate with an access point (AP). Users arrive to the system at random and be activated for a certain period of time slots and then disappear from the system. Under such dynamic network environment, we propose a distributed multichannel access protocol based on multi-agent reinforcement learning (RL) to improve both throughput and fairness between active users. Unlike the previous approaches adjusting channel access probabilities at each time slot, the proposed RL algorithm deterministically selects a set of channel access policies for several consecutive time slots. To effectively reduce the complexity of the proposed RL algorithm, we adopt a branching dueling Q-network architecture and propose an efficient training methodology for producing proper Q-values over time-varying user sets. We perform extensive simulations on realistic traffic environments and demonstrate that the proposed online learning improves both throughput and fairness compared to the conventional RL approaches and centralized scheduling policies.