Off-Policy Integral Reinforcement Learning Method to Solve Nonlinear Continuous-Time Multiplayer Nonzero-Sum Games

• Published in: IEEE transaction on neural networks and learning systems Vol. 28; no. 3; pp. 704 - 713
• Main Authors: Ruizhuo Song, Lewis, Frank L., Qinglai Wei
• Format: Journal Article
• Language: English
• Published: United States IEEE 01.03.2017; The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
• Subjects: Adaptive critic designs; adaptive dynamic programming (ADP); Algorithms; approximate dynamic programming; Computer simulation; Convergence; Feedback control; Games; Heuristic algorithms; integral reinforcement learning (IRL); Integrals; Iterative algorithms; Learning; Machine learning; Markov analysis; Mathematical model; Nash equilibrium; nonlinear systems; nonzero sum (NZS); off-policy; Optimal control; Performance indices; Reinforcement; Stability analysis; System dynamics
• ISSN: 2162-237X; 2162-2388; 2162-2388
• DOI: 10.1109/TNNLS.2016.2582849

This paper establishes an off-policy integral reinforcement learning (IRL) method to solve nonlinear continuous-time (CT) nonzero-sum (NZS) games with unknown system dynamics. The IRL algorithm is presented to obtain the iterative control and off-policy learning is used to allow the dynamics to be completely unknown. Off-policy IRL is designed to do policy evaluation and policy improvement in the policy iteration algorithm. Critic and action networks are used to obtain the performance index and control for each player. The gradient descent algorithm makes the update of critic and action weights simultaneously. The convergence analysis of the weights is given. The asymptotic stability of the closed-loop system and the existence of Nash equilibrium are proved. The simulation study demonstrates the effectiveness of the developed method for nonlinear CT NZS games with unknown system dynamics.