Joint Beamforming, Power Allocation, and Splitting Control for SWIPT-Enabled IoT Networks with Deep Reinforcement Learning and Game Theory

• Published in: Sensors (Basel, Switzerland) Vol. 22; no. 6; p. 2328
• Main Authors: Liu, JainShing, Lin, Chun-Hung Richard, Hu, Yu-Chen, Donta, Praveen Kumar
• Format: Journal Article
• Language: English
• Published: Switzerland MDPI AG 17.03.2022; MDPI
• Subjects: Algorithms; beamforming; Business metrics; Communication; Convex analysis; Deep learning; deep reinforcement learning; Energy consumption; Game theory; Internet of Things; joint optimization; Methods; multi-resource allocation; Optimization; power control; Wireless networks; joint optimization; multi-resource allocation; game theory; energy harvesting; power control; deep reinforcement learning; beamforming; IoT
• DOI: 10.3390/s22062328
• ISSN: 1424-8220

Future wireless networks promise immense increases on data rate and energy efficiency while overcoming the difficulties of charging the wireless stations or devices in the Internet of Things (IoT) with the capability of simultaneous wireless information and power transfer (SWIPT). For such networks, jointly optimizing beamforming, power control, and energy harvesting to enhance the communication performance from the base stations (BSs) (or access points (APs)) to the mobile nodes (MNs) served would be a real challenge. In this work, we formulate the joint optimization as a mixed integer nonlinear programming (MINLP) problem, which can be also realized as a complex multiple resource allocation (MRA) optimization problem subject to different allocation constraints. By means of deep reinforcement learning to estimate future rewards of actions based on the reported information from the users served by the networks, we introduce single-layer MRA algorithms based on deep Q-learning (DQN) and deep deterministic policy gradient (DDPG), respectively, as the basis for the downlink wireless transmissions. Moreover, by incorporating the capability of data-driven DQN technique and the strength of noncooperative game theory model, we propose a two-layer iterative approach to resolve the NP-hard MRA problem, which can further improve the communication performance in terms of data rate, energy harvesting, and power consumption. For the two-layer approach, we also introduce a pricing strategy for BSs or APs to determine their power costs on the basis of social utility maximization to control the transmit power. Finally, with the simulated environment based on realistic wireless networks, our numerical results show that the two-layer MRA algorithm proposed can achieve up to 2.3 times higher value than the single-layer counterparts which represent the data-driven deep reinforcement learning-based algorithms extended to resolve the problem, in terms of the utilities designed to reflect the trade-off among the performance metrics considered.