Collaborative Intelligent Resource Trading for RAN Slicing: A Federated Policy Distillation Approach

• Published in: 2023 International Conference on Networks, Communications and Intelligent Computing (NCIC) pp. 73 - 77
• Main Authors: Ayepah-Mensah, Daniel, Sun, Guolin
• Format: Conference Proceeding
• Language: English
• Published: IEEE 17.11.2023
• Subjects: Collaboration; Computational modeling; Distributed databases; Federated deep reinforcement learning; Markov decision processes; Mutual distillation; Pricing; Privacy-preserving; Radio Access Network slicing; Resource management; Resource trading; Robustness
• DOI: 10.1109/NCIC61838.2023.00018

In Radio Access Network (RAN), the sharing of resources can be modeled as a trading process in which multiple Mobile Virtual Network Operators (MVNOs) buy and sell resources according to their needs. This process can be competitive, with each MVNO strategically pricing and managing resources to maximize utility. Despite the dynamic nature of RAN slicing, deep-reinforcement learning (DRL) solutions often perform best but can be impractical due to their centralized nature and the need for full cooperation. These methods may face inconsistency due to MVNO heterogeneity, leading to imbalanced data distribution and potential selfishness, complicating optimal solution achievement. This paper proposes a collaborative intelligent framework for resource trading based on the Federated Deep Reinforcement Learning framework with Mutual Policy Distillation (FDRL-MPD), which enables MVNOs to collaborate and learn personalized trading models. Furthermore, we proposed a reward-shaping mechanism based on the proxy policy optimization (PPO) algorithm for local resource trading. Simulations performed with several MVNOs confirm the effectiveness of the proposed framework, especially concerning the algorithm's robustness to not independent and identically distributed data.