Optimal control with deep reinforcement learning for shunt compensations to enhance voltage stability

• Published in: 2020 5th Asia Conference on Power and Electrical Engineering (ACPEE) pp. 398 - 403
• Main Authors: Cao, Shang, Liao, Shiwu, Wang, Shaorong, Luo, Xiaotong
• Format: Conference Proceeding
• Language: English
• Published: IEEE 01.06.2020
• Subjects: Aerospace electronics; Complexity theory; Deep Reinforcement Learning; Discrete Action Space; Optimal control; Parallel Reactive Compensation; Power system stability; Reinforcement learning; Stability analysis; Static voltage stability; Voltage control
• DOI: 10.1109/ACPEE48638.2020.9136271

Due to the severe complexity caused by the expanding interconnection and the increasing penetration of power electronic devices, optimally control the shunt compensations is facing great challenges. Deep reinforcement learning (DRL) has tremendous potential in handling the complexity and nonlinearity in the power systems. But traditional DRL methods are difficult or even impossible to apply to tasks with large discrete action space, while the shunt compensation control exactly has extremely large discrete action space. To adopt DRL to the optimal control of shunt compensations, this paper proposed a new voltage control method based on the Wolpertinger Architecture of DRL. The proposed method embeds discrete action in a continuous space, use DDPG to produce a continuous action, and then use proximity algorithm to search the final action, which reduces the time complexity of action lookup. The proposed method was tested on IEEE 118 bus system with 6 20 discrete actions, and shows good performance enhancing the voltage stability.