Secure Control for Markov Jump Cyber-Physical Systems Subject to Malicious Attacks: A Resilient Hybrid Learning Scheme

• Published in: IEEE transactions on cybernetics Vol. 54; no. 11; pp. 7068 - 7079
• Main Authors: Shen, Hao, Wang, Yun, Wu, Jiacheng, Park, Ju H., Wang, Jing
• Format: Journal Article
• Language: English
• Published: United States IEEE 01.11.2024
• Subjects: Games; Hybrid learning; Markov jump cyber-physical systems (MJCPSs); Optimal control; Q-learning; reinforcement learning (RL); resilient control; Riccati equations; Sensors; System dynamics; zero-sum game
• ISSN: 2168-2267; 2168-2275; 2168-2275
• DOI: 10.1109/TCYB.2024.3448407

This article focuses on solving the secure control problem by developing a novel resilient hybrid learning scheme for discrete-time Markov jump cyber-physical systems with malicious attacks. Within the zero-sum game framework, the secure control problem is converted into solving a set of game coupled algebraic Riccati equations. However, it contains the coupling terms arising from the Markov jump parameters, which are difficult to solve. To address this issue, we propose a framework for parallel reinforcement learning. Thereafter, a model-based resilient hybrid learning scheme is first designed to obtain the optimal policies, where the system dynamics are required during the learning process. Furthermore, a novel online model-free resilient hybrid learning scheme combining the advantages of value iteration and policy iteration is proposed without using the system dynamics. Besides, the convergence of the proposed hybrid learning schemes is discussed. Eventually, the effectiveness of the designed algorithms is demonstrated with the inverted pendulum model.