Federated Deep Reinforcement Learning for Task Offloading in Digital Twin Edge Networks

• Published in: IEEE transactions on network science and engineering Vol. 11; no. 3; pp. 2849 - 2863
• Main Authors: Dai, Yueyue, Zhao, Jintang, Zhang, Jing, Zhang, Yan, Jiang, Tao
• Format: Journal Article
• Language: English
• Published: Piscataway IEEE 01.05.2024; The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
• Subjects: Base stations; Computation offloading; Computational modeling; Configurations; Deep learning; Deep reinforcement learning; Digital twin edge networks; Digital twins; Edge computing; federated deep reinforcement learning; Federated learning; Links; Mobile computing; Optimization; Resource management; Servers; Task analysis; task offloading; Training
• ISSN: 2327-4697; 2334-329X
• DOI: 10.1109/TNSE.2024.3350710

Digital twin edge networks provide a new paradigm that combines mobile edge computing (MEC) and digital twins to improve network performance and reduce communication cost by utilizing digital twin models of physical objects. The construction of digital twin models requires powerful computing ability. However, the distributed devices with limited computing resources cannot complete high-fidelity digital twin construction. Moreover, weak communication links between these devices may hinder the potential of digital twins. To address these issues, we propose a two-layer digital twin edge network, in which the physical network layer offloads training tasks using passive reflecting links, and the digital twin layer establishes a digital twin model to record the dynamic states of physical components. We then formulate a system cost minimization problem to jointly optimize task offloading, configurations of passive reflecting links, and computing resources. Finally, we design a federated deep reinforcement learning (DRL) scheme to solve the problem, where local agents train offloading decisions and global agents optimize the allocation of edge computing resources and configurations of passive reflecting elements. Numerical results show the effectiveness of the proposed federated DRL and it can reduce the system cost by up to 67.1% compared to the benchmarks.