Distributed Task Migration Optimization in MEC by Extending Multi-Agent Deep Reinforcement Learning Approach

• Published in: IEEE transactions on parallel and distributed systems Vol. 32; no. 7; pp. 1603 - 1614
• Main Authors: Liu, Chubo, Tang, Fan, Hu, Yikun, Li, Kenli, Tang, Zhuo, Li, Keqin
• Format: Journal Article
• Language: English
• Published: New York IEEE 01.07.2021; The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
• Subjects: Algorithms; Cloud computing; Completion time; Computational modeling; Decision making; Edge computing; Energy; Energy budget; Energy consumption; Machine learning; Markov chains; Markov processes; Mobile computing; mobile edge computing; mobility; multi-agent reinforcement learning; Multiagent systems; Optimization; Quality of service; Reinforcement learning; Task analysis; task migration
• ISSN: 1045-9219; 1558-2183
• DOI: 10.1109/TPDS.2020.3046737

Closer to mobile users geographically, mobile edge computing (MEC) can provide some cloud-like capabilities to users more efficiently. This enables it possible for resource-limited mobile users to offload their computation-intensive and latency-sensitive tasks to MEC nodes. For its great benefits, MEC has drawn wide attention and extensive works have been done. However, few of them address task migration problem caused by distributed user mobility, which can't be ignored with quality of service (QoS) consideration. In this article, we study task migration problem and try to minimize the average completion time of tasks under migration energy budget. There are multiple independent users and the movement of each mobile user is memoryless with a sequential decision-making process, thus reinforcement learning algorithm based on Markov chain model is applied with low computation complexity. To further facilitate cooperation among users, we devise a distributed task migration algorithm based on counterfactual multi-agent (COMA) reinforcement learning approach to solve this problem. Extensive experiments are carried out to assess the performance of this distributed task migration algorithm. Compared with no migrating (NM) and single-agent actor-critic (AC) algorithms, the proposed distributed task migration algorithm can achieve up 30-50 percent reduction about average completion time.