Multiple Reconfigurable Intelligent Surfaces Aided Vehicular Edge Computing Networks: A MAPPO-Based Approach

• Published in: IEEE transactions on vehicular technology Vol. 73; no. 11; pp. 17496 - 17509
• Main Authors: Ning, Xiangrui, Zeng, Ming, Hua, Meng, Fei, Zesong
• Format: Journal Article
• Language: English
• Published: New York IEEE 01.11.2024; The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
• Subjects: Algorithms; Communication networks; Computation offloading; Delays; Edge computing; Machine learning; Mixed integer; multi-agent reinforcement learning; Multiagent systems; Optimization; Radio equipment; Radio transmission; Reconfigurable intelligent surfaces; Reinforcement learning; Resource management; service migration; Throughput; Vehicular ad hoc networks; Vehicular edge computing; Virtual environments; Wireless communication; Wireless networks
• ISSN: 0018-9545; 1939-9359
• DOI: 10.1109/TVT.2024.3419554

Reconfigurable intelligent surface (RIS) is envisioned as a new technology to improve the quality-of-service in vehicular edge computing (VEC) networks due to its ability to change the wireless radio propagation environment. In this paper, we study multi-RIS-assisted VEC networks, where vehicle user equipments (VUEs) can offload tasks to nearby base stations (BSs) which offer efficient computation edge services (ESs). Meanwhile, the individual virtual machine (VM), which is defined as a software clone of the VUE's service environment containing the profile and application to run the VUE's tasks, need to be migrated to the same ES for offloaded task completion. Accordingly, we formulate a throughput maximization problem for multi-RIS-assisted VEC networks via jointly optimizing the selected ESs, the deployment locations of RISs, and reflection matrices of RISs, subject to the maximum tolerable delay. To solve the non-convex mixed-integer nonlinear optimization problem, we propose an efficient algorithm based on multi-agent proximal policy optimization (MAPPO) with the centralized training and decentralized execution (CTDE) framework, where two types of heterogeneous agents are considered. In particular, several tricks such as reward normalization, orthogonal initialization, and learning rate decay are adopted to accelerate the convergence of the proposed algorithm. Numerical simulation results show that the throughput obtained by the proposed MAPPO-based scheme outperforms that obtained by the scheme without multi-RIS 26.41% and that obtained by the scheme without service migration 23.65%, respectively. Moreover, compared to other conventional multi-agent reinforcement learning (MARL) algorithms, the proposed algorithm converges faster.