Adaptive Reinforcement Learning Neural Network Control for Uncertain Nonlinear System With Input Saturation

• Published in: IEEE transactions on cybernetics Vol. 50; no. 8; pp. 3433 - 3443
• Main Authors: Bai, Weiwei, Zhou, Qi, Li, Tieshan, Li, Hongyi
• Format: Journal Article
• Language: English
• Published: United States IEEE 01.08.2020; The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
• Subjects: Actuators; Adaptive control; Adaptive systems; Algorithms; Artificial neural networks; Closed loop systems; Computer simulation; Convergence; Discrete time systems; Feedback control; input saturation; Learning; Machine Learning; multigradient recursive (MGR); Network control; Neural networks; neural networks (NNs); Neural Networks, Computer; Nonlinear control; Nonlinear Dynamics; Nonlinear systems; Optimal control; Radial basis function; Reinforcement learning; Saturation; Signal Processing, Computer-Assisted; Systems stability
• ISSN: 2168-2267; 2168-2275; 2168-2275
• DOI: 10.1109/TCYB.2019.2921057

In this paper, an adaptive neural network (NN) control problem is investigated for discrete-time nonlinear systems with input saturation. Radial-basis-function (RBF) NNs, including critic NNs and action NNs, are employed to approximate the utility functions and system uncertainties, respectively. In the previous works, a gradient descent scheme is applied to update weight vectors, which may lead to local optimal problem. To circumvent this problem, a multigradient recursive (MGR) reinforcement learning scheme is proposed, which utilizes both the current gradient and the past gradients. As a consequence, the MGR scheme not only eliminates the local optimal problem but also guarantees faster convergence rate than the gradient descent scheme. Moreover, the constraint of actuator input saturation is considered. The closed-loop system stability is developed by using the Lyapunov stability theory, and it is proved that all the signals in the closed-loop system are semiglobal uniformly ultimately bounded (SGUUB). Finally, the effectiveness of the proposed approach is further validated via some simulation results.