A Homotopy Method for Continuous-Time Model-Free LQR Control Based on Policy Iteration

In recent years, reinforcement learning control theory has been well developed. However, model-free value iteration needs many iterations to achieve the desired precision, and model-free policy iteration requires an initial stabilizing control policy. It is significant to propose a fast model-free a...

Full description

Saved in:
Bibliographic Details
Published inIEEE/CAA journal of automatica sinica Vol. 12; no. 8; pp. 1673 - 1682
Main Authors Fan, Wenwu, Xiong, Junlin
Format Journal Article
LanguageEnglish
Published Chinese Association of Automation (CAA) 01.08.2025
Subjects
Approximation algorithms
Computational complexity
Convergence
Homotopy path
initial stabilizing control policy
linear quadratic control
Linear systems
Mathematical models
Optimal control
policy iteration
Regulators
Reinforcement learning
State feedback
Symmetric matrices
Online AccessGet full text

Cover

More Information
Summary:In recent years, reinforcement learning control theory has been well developed. However, model-free value iteration needs many iterations to achieve the desired precision, and model-free policy iteration requires an initial stabilizing control policy. It is significant to propose a fast model-free algorithm to solve the continuous-time linear quadratic control problem without an initial stabilizing control policy. In this paper, we construct a homotopy path on which each point corresponds to an linear quadratic regulator problem. Based on policy iteration, model-based and model-free homotopy algorithms are proposed to solve the optimal control problem of continuous-time linear systems along the homotopy path. Our algorithms are speeded up using first-order differential information and do not require an initial stabilizing control policy. Finally, several practical examples are used to illustrate our results.
ISSN:2329-9266
2329-9274
DOI:10.1109/JAS.2025.125132