Federated reinforcement learning for Short-Time scale operation of Wind-Solar-Thermal power network with nonconvex models

• Published in: International journal of electrical power & energy systems Vol. 158; p. 109980
• Main Authors: Zou, Yao, Wang, Qianggang, Xia, Qinqin, Chi, Yuan, Lei, Chao, Zhou, Niancheng
• Format: Journal Article
• Language: English
• Published: Elsevier Ltd 01.07.2024; Elsevier
• Subjects: Federated reinforcement learning; Power sources scheduling; Renewable energy; Thermal power unit; Power sources scheduling; Renewable energy; Thermal power unit; Federated reinforcement learning
• ISSN: 0142-0615; 1879-3517
• DOI: 10.1016/j.ijepes.2024.109980

•Power sources operation for wind-solar-thermal power system with nonconvex models.•Combination of the federated reinforcement learning (FRL) with the model-based method.•Grid-connected renewable energy and thermal power at a bus are aggregated as a VPP.•Private operation with limited but effective information exchange.•Customized FRL algorithm with efficient agent training and privacy preservation. To schedule power sources operated by different entities in a short-time scale considering nonconvex generation cost and deep peak regulation (DPR) service constraints, this paper proposes an FRL-based multiple power sources coordination framework in wind-solar-thermal power network. In the studied power transmission network (TN), renewable energy sources and thermal power units connected to the same bus are aggregated as a wind-solar-thermal virtual power plant (WSTVPP). The transmission system operator (TSO) sends dispatch instructions to each WSTVPP by optimal power flow program, and allocates the cost of DPR service in TN. Based on the dispatch instruction, the internal power sources of each WSTVPP are scheduled by its local center control agent to achieve local economic operation while maximizing the overall DPR service revenue for the WSTVPP from the auxiliary service market. The multiple WSTVPPs operation is modeled as a partially observable Markov decision process, and solved by a designed FRL algorithm. The FRL algorithm employs a global neural network (NN) model for coordination, heterogeneous local NN models and data to efficiently train each WSTVPP control agent with individual objectives for handling multiple power sources scheduling in TN while preserving local privacy. Numerical studies validate the effectiveness of the proposed framework for handling the short-time scale power sources operation with nonconvex constraints.