Policy Optimization of the Power Allocation Algorithm Based on the Actor–Critic Framework in Small Cell Networks

A practical solution to the power allocation problem in ultra-dense small cell networks can be achieved by using deep reinforcement learning (DRL) methods. Unlike traditional algorithms, DRL methods are capable of achieving low latency and operating without the need for global real-time channel stat...

Full description

Saved in:
Bibliographic Details
Published inMathematics (Basel) Vol. 11; no. 7; p. 1702
Main Authors Chen, Haibo, Huang, Zhongwei, Zhao, Xiaorong, Liu, Xiao, Jiang, Youjun, Geng, Pinyong, Yang, Guang, Cao, Yewen, Wang, Deqiang
Format Journal Article
LanguageEnglish
Published Basel MDPI AG 01.04.2023
Subjects
actor–critic
Algorithms
Deep learning
deep reinforcement learning
Efficiency
Energy consumption
Energy use
Mathematical optimization
Methods
Mobile communication systems
Network latency
Optimization
Optimization algorithms
power allocation
Random variables
Real time
Resource allocation
Wireless communication systems
Wireless networks
China
Online AccessGet full text

Cover

More Information
Summary:A practical solution to the power allocation problem in ultra-dense small cell networks can be achieved by using deep reinforcement learning (DRL) methods. Unlike traditional algorithms, DRL methods are capable of achieving low latency and operating without the need for global real-time channel state information (CSI). Based on the actor–critic framework, we propose a policy optimization of the power allocation algorithm (POPA) for small cell networks in this paper. The POPA adopts the proximal policy optimization (PPO) algorithm to update the policy, which has been shown to have stable exploration and convergence effects in our simulations. Thanks to our proposed actor–critic architecture with distributed execution and centralized exploration training, the POPA can meet real-time requirements and has multi-dimensional scalability. Through simulations, we demonstrate that the POPA outperforms existing methods in terms of spectral efficiency. Our findings suggest that the POPA can be of practical value for power allocation in small cell networks.
ISSN:2227-7390
2227-7390
DOI:10.3390/math11071702