Multi-Agent Reinforcement Learning Based Resource Management in MEC- and UAV-Assisted Vehicular Networks

• Published in: IEEE journal on selected areas in communications Vol. 39; no. 1; pp. 131 - 141
• Main Authors: Peng, Haixia, Shen, Xuemin
• Format: Journal Article
• Language: English
• Published: New York IEEE 01.01.2021; The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
• Subjects: Decisions; Delays; Edge computing; Learning; Mobile computing; multi-access edge computing; multi-agent DDPG; multi-dimensional resource management; Multiagent systems; Optimization; Quality of service; Resource allocation; Resource management; Servers; Task analysis; Training; unmanned aerial vehicle; Unmanned aerial vehicles; Vehicles; Vehicular networks; Wireless communication
• ISSN: 0733-8716; 1558-0008
• DOI: 10.1109/JSAC.2020.3036962

In this paper, we investigate multi-dimensional resource management for unmanned aerial vehicles (UAVs) assisted vehicular networks. To efficiently provide on-demand resource access, the macro eNodeB and UAV, both mounted with multi-access edge computing (MEC) servers, cooperatively make association decisions and allocate proper amounts of resources to vehicles. Since there is no central controller, we formulate the resource allocation at the MEC servers as a distributive optimization problem to maximize the number of offloaded tasks while satisfying their heterogeneous quality-of-service (QoS) requirements, and then solve it with a multi-agent deep deterministic policy gradient (MADDPG)-based method. Through centrally training the MADDPG model offline, the MEC servers, acting as learning agents, then can rapidly make vehicle association and resource allocation decisions during the online execution stage. From our simulation results, the MADDPG-based method can converge within 200 training episodes, comparable to the single-agent DDPG (SADDPG)-based one. Moreover, the proposed MADDPG-based resource management scheme can achieve higher delay/QoS satisfaction ratios than the SADDPG-based and random schemes.