Adaptive optimal control for continuous-time linear systems based on policy iteration

• Published in: Automatica (Oxford) Vol. 45; no. 2; pp. 477 - 484
• Main Authors: Vrabie, D., Pastravanu, O., Abu-Khalaf, M., Lewis, F.L.
• Format: Journal Article
• Language: English
• Published: Kidlington Elsevier Ltd 01.02.2009; Elsevier
• Subjects: Adaptative systems; Adaptive control; Adaptive critics; Applied sciences; Computer science; control theory; systems; Control theory. Systems; Exact sciences and technology; LQR; Optimal control; Policy iterations; Adaptive critics; Policy iterations; LQR; Adaptive control; State feedback; Feedback regulation; Continuous time; Linear time; Internal model principle; Dynamical system; Linear control; Modeling; Closed feedback; LQ control; Continuous control; Optimal control; Algebraic equation; Infinite horizon; Riccati equation; Dynamic model
• ISSN: 0005-1098; 1873-2836
• DOI: 10.1016/j.automatica.2008.08.017

In this paper we propose a new scheme based on adaptive critics for finding online the state feedback, infinite horizon, optimal control solution of linear continuous-time systems using only partial knowledge regarding the system dynamics. In other words, the algorithm solves online an algebraic Riccati equation without knowing the internal dynamics model of the system. Being based on a policy iteration technique, the algorithm alternates between the policy evaluation and policy update steps until an update of the control policy will no longer improve the system performance. The result is a direct adaptive control algorithm which converges to the optimal control solution without using an explicit, a priori obtained, model of the system internal dynamics. The effectiveness of the algorithm is shown while finding the optimal-load-frequency controller for a power system.