Discrete-Time Optimal Control via Local Policy Iteration Adaptive Dynamic Programming

• Published in: IEEE transactions on cybernetics Vol. 47; no. 10; pp. 3367 - 3379
• Main Authors: Wei, Qinglai, Liu, Derong, Lin, Qiao, Song, Ruizhuo
• Format: Journal Article
• Language: English
• Published: United States IEEE 01.10.2017; The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
• Subjects: Adaptive algorithms; Adaptive control; Adaptive critic designs; adaptive dynamic programming (ADP); Algorithms; approximate dynamic programming; Computer simulation; Convergence; Discrete-event systems; Dynamic programming; Heuristic algorithms; Iterative algorithms; Iterative methods; local policy iteration; Markov analysis; neuro-dynamic programming; Nonlinear systems; OptimAL control; Optimization; Servers; Time optimal control
• ISSN: 2168-2267; 2168-2275
• DOI: 10.1109/TCYB.2016.2586082

In this paper, a discrete-time optimal control scheme is developed via a novel local policy iteration adaptive dynamic programming algorithm. In the discrete-time local policy iteration algorithm, the iterative value function and iterative control law can be updated in a subset of the state space, where the computational burden is relaxed compared with the traditional policy iteration algorithm. Convergence properties of the local policy iteration algorithm are presented to show that the iterative value function is monotonically nonincreasing and converges to the optimum under some mild conditions. The admissibility of the iterative control law is proven, which shows that the control system can be stabilized under any of the iterative control laws, even if the iterative control law is updated in a subset of the state space. Finally, two simulation examples are given to illustrate the performance of the developed method.