Robust Proactive Power Smoothing Control of PV Systems Based on Deep Reinforcement Learning

• Published in: IEEE transactions on sustainable energy Vol. 14; no. 3; pp. 1585 - 1598
• Main Authors: Chen, Xiaoyang, Xu, Xu, Wang, Jia, Fang, Lurui, Du, Yang, Lim, Eng Gee, Ma, Jieming
• Format: Journal Article
• Language: English
• Published: Piscataway IEEE 01.07.2023; The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
• Subjects: Compensators; Control tasks; Deep learning; deep reinforcement learning (RL); Forecasting; Intermittency; Photovoltaic cells; Photovoltaics; Photovoltaics (PV); Power smoothing; Predictive models; proactive power smoothing; Random access memory; Reinforcement; Robust control; Smoothing; Smoothing methods; solar forecasting; Stochasticity; Training
• ISSN: 1949-3029; 1949-3037
• DOI: 10.1109/TSTE.2023.3239852

Efficient integration of photovoltaic (PV) energy into the power grid calls for a robust regulation of its intermittency. At present, the solar forecasting-assisted proactive power smoothing control (PPSC) has shown superiority in handling such PV intermittency due to its battery-less operations. However, the implementation of PPSC is largely dependent on the quality of solar forecasts, i.e, high accuracy over a long horizon, which has seriously limited its extensive application in practice. In this context, this paper proposes a novel PPSC method based on deep reinforcement learning (RL). In addition to the actor-critic structures, a new module, namely, compensator is developed to tackle the problems of sparse reward and state transition stochasticity that are typically associated with the PPSC control task. On top of it, a novel scenario recognized experience replay (SRER) is devised to deal with the data distribution mismatching issue in PPSC. The effectiveness of the proposed method is verified using real-world data from a solar measurement grid. Empirical studies show that compared with the conventional PPSC method, the proposed method can achieve more robust smoothing performance under various forecasting scenarios, rendering it more applicable to practical PV system operations.