Distributed Q-Learning-Based Voltage Restoration Algorithm in Isolated AC Microgrids Subject to Input Saturation

• Published in: IEEE transactions on industry applications Vol. 60; no. 4; pp. 5447 - 5459
• Main Authors: Lin, Shih-Wen, Chu, Chia-Chi
• Format: Journal Article
• Language: English
• Published: New York IEEE 01.07.2024; The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
• Subjects: Algorithms; Bellman theory; consensus algorithm; Decentralized control; distributed control; Distributed generation; Electric potential; Exact solutions; Feedback; Heuristic algorithms; Internet of Things; Machine learning; Mathematical models; Microgrid; modified algebraic Riccati equation; multi-agent reinforcement learning; plug-and-play; Q-learning; Recursive functions; Restoration; Riccati equation; Riccati equations; saturation limit; Tracking control; Tracking errors; Voltage; Voltage control; voltage restoration
• ISSN: 0093-9994; 1939-9367
• DOI: 10.1109/TIA.2024.3379965

A model-free data-driven Q-learning-based distributed control is proposed for achieving autonomous voltage restoration in isolated AC microgrids (MGs) subject to input saturation. First, by defining the control objective in terms of local neighborhood tracking errors, the voltage restoration problem can be solved by the distributed pinning-based consensus problem. To address the effect of input signal saturation in each distributed generator (DG), the low gain feedback method is considered to obtain these feedback gains. Since these feedback gain matrices are obtained by solving the modified algebraic Riccati equation (MARE) which needs the complete knowledge of DG dynamics, an iterative model-free data-driven Q-learning algorithm is presented. A Q-learning function and a Q-learning Bellman equation are defined for finding these feedback gain matrices. Finally, based on recursive least square techniques, an iterative Q-learning algorithm is proposed for achieving autonomous voltage restoration. To validate the performance of the proposed method, simulations of two isolated AC MG are performed. Simulation results demonstrate that the acquired feedback gains closely align with these analytical solutions of the MARE. Furthermore, the proposed model-free distributed Q-learning method remains its effectiveness even under model uncertainty and plug-and-play operations of DGs.