Reinforcement Learning-Based Optimal Stabilization for Unknown Nonlinear Systems Subject to Inputs With Uncertain Constraints

• Published in: IEEE transaction on neural networks and learning systems Vol. 31; no. 10; pp. 4330 - 4340
• Main Authors: Zhao, Bo, Liu, Derong, Luo, Chaomin
• Format: Journal Article
• Language: English
• Published: United States IEEE 01.10.2020; The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
• Subjects: Actuators; Adaptive dynamic programming (ADP); Algorithms; Artificial neural networks; Control algorithms; Control theory; Distance learning; Feedback control; Feedforward control; Feedforward systems; Learning; Machine learning; Neural networks; neural networks (NNs); Nonlinear systems; Observers; Optimal control; Reinforcement; reinforcement learning (RL); Saturation; Stability analysis; System dynamics; uncertain input constraints; unknown nonlinear systems
• DOI: 10.1109/TNNLS.2019.2954983
• ISSN: 2162-237X; 2162-2388

This article presents a novel reinforcement learning strategy that addresses an optimal stabilizing problem for unknown nonlinear systems subject to uncertain input constraints. The control algorithm is composed of two parts, i.e., online learning optimal control for the nominal system and feedforward neural networks (NNs) compensation for handling uncertain input constraints, which are considered as the saturation nonlinearities. Integrating the input-output data and recurrent NN, a Luenberger observer is established to approximate the unknown system dynamics. For nominal systems without input constraints, the online learning optimal control policy is derived by solving Hamilton-Jacobi-Bellman equation via a critic NN alone. By transforming the uncertain input constraints to saturation nonlinearities, the uncertain input constraints can be compensated by employing a feedforward NN compensator. The convergence of the closed-loop system is guaranteed to be uniformly ultimately bounded by using the Lyapunov stability analysis. Finally, the effectiveness of the developed stabilization scheme is illustrated by simulation studies.