Deep Reinforcement Learning for Smart Grid Operations: Algorithms, Applications, and Prospects

• Published in: Proceedings of the IEEE Vol. 111; no. 9; pp. 1055 - 1096
• Main Authors: Li, Yuanzheng, Yu, Chaofan, Shahidehpour, Mohammad, Yang, Tao, Zeng, Zhigang, Chai, Tianyou
• Format: Journal Article
• Language: English
• Published: New York IEEE 01.09.2023; The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
• Subjects: Algorithms; Artificial intelligence; Computer applications; Deep learning; Deep reinforcement learning (DRL); Electric power systems; electricity market; Electricity supply industry deregulation; Flexible electronics; Literature reviews; Machine learning; Mathematical models; operational control; optimal dispatch; Optimization; Power dispatch; Power markets; Power system control; Power system stability; Reinforcement learning; Renewable energy sources; Smart grid; smart grid (SG); Smart grids; Surveys; Uncertainty
• ISSN: 0018-9219; 1558-2256
• DOI: 10.1109/JPROC.2023.3303358

With the increasing penetration of renewable energy and flexible loads in smart grids, a more complicated power system with high uncertainty is gradually formed, which brings about great challenges to smart grid operations. Traditional optimization methods usually require accurate mathematical models and parameters and cannot deal well with the growing complexity and uncertainty. Fortunately, the widespread popularity of advanced meters makes it possible for smart grid to collect massive data, which offers opportunities for data-driven artificial intelligence methods to address the optimal operation and control issues. Therein, deep reinforcement learning (DRL) has attracted extensive attention for its excellent performance in operation problems with high uncertainty. To this end, this article presents a comprehensive literature survey on DRL and its applications in smart grid operations. First, a detailed overview of DRL, from fundamental concepts to advanced models, is conducted in this article. Afterward, we review various DRL techniques as well as their extensions developed to cope with emerging issues in the smart grid, including optimal dispatch, operational control, electricity market, and other emerging areas. In addition, an application-oriented survey of DRL in smart grid is presented to identify difficulties for future research. Finally, essential challenges, potential solutions, and future research directions concerning the DRL applications in smart grid are also discussed.