A Deep Reinforcement Learning Based Offloading Game in Edge Computing

• Published in: IEEE transactions on computers Vol. 69; no. 6; pp. 883 - 893
• Main Authors: Zhan, Yufeng, Guo, Song, Li, Peng, Zhang, Jiang
• Format: Journal Article
• Language: English
• Published: New York IEEE 01.06.2020; The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
• Subjects: Algorithms; Computation offloading; Computational modeling; Computer simulation; Decision theory; Deep learning; deep reinforcement learning (DRL); Edge computing; Game theory; Games; Markov processes; Nash equilibrium; partially observable Markov decision process (POMDP); Reinforcement learning; Servers; Task analysis
• ISSN: 0018-9340; 1557-9956
• DOI: 10.1109/TC.2020.2969148

Edge computing is a new paradigm to provide strong computing capability at the edge of pervasive radio access networks close to users. A critical research challenge of edge computing is to design an efficient offloading strategy to decide which tasks can be offloaded to edge servers with limited resources. Although many research efforts attempt to address this challenge, they need centralized control, which is not practical because users are rational individuals with interests to maximize their benefits. In this article, we study to design a decentralized algorithm for computation offloading, so that users can independently choose their offloading decisions. Game theory has been applied in the algorithm design. Different from existing work, we address the challenge that users may refuse to expose their information about network bandwidth and preference. Therefore, it requires that our solution should make the offloading decision without such knowledge. We formulate the problem as a partially observable Markov decision process (POMDP), which is solved by a policy gradient deep reinforcement learning (DRL) based approach. Extensive simulation results show that our proposal significantly outperforms existing solutions.