Lifelong Learning-Based Optimal Trajectory Tracking Control of Constrained Nonlinear Affine Systems Using Deep Neural Networks

• Published in: IEEE transactions on cybernetics Vol. 54; no. 12; pp. 7133 - 7146
• Main Authors: Ganie, Irfan, Jagannathan, Sarangapani
• Format: Journal Article
• Language: English
• Published: United States IEEE 01.12.2024
• Subjects: Artificial neural networks; Lifelong learning (LL); Multi-layer neural network; multilayer neural network (MNN); Multitasking; Optimal control; reinforcement learning; singular value decomposition (SVD); Trajectory; Trajectory tracking; Vectors
• ISSN: 2168-2267; 2168-2275; 2168-2275
• DOI: 10.1109/TCYB.2024.3405354

This article presents a novel lifelong integral reinforcement learning (LIRL)-based optimal trajectory tracking scheme using the multilayer (MNN) or deep neural network (Deep NN) for the uncertain nonlinear continuous-time (CT) affine systems subject to state constraints. A critic MNN, which approximates the value function, and a second NN identifier are together used to generate the optimal control policies. The weights of the critic MNN are tuned online using a novel singular value decomposition (SVD)-based method, which can be extended to MNN with the N-hidden layers. Moreover, an online lifelong learning (LL) scheme is incorporated with the critic MNN to mitigate the problem of catastrophic forgetting in the multitasking systems. Additionally, the proposed optimal framework addresses state constraints by utilizing a time-varying barrier function (TVBF). The uniform ultimate boundedness (UUB) of the overall closed-loop system is shown using the Lyapunov stability analysis. A two-link robotic manipulator that compares to recent literature shows a 47% total cost reduction, demonstrating the effectiveness of the proposed method.