Q-learning for continuous-time linear systems: A model-free infinite horizon optimal control approach

In this paper we propose an online Q-learning algorithm to solve the infinite-horizon optimal control problem of a linear time invariant system with completely uncertain/unknown dynamics. We first formulate the Q-function by using the Hamiltonian and the optimal cost. An integral reinforcement learn...

Full description

Saved in:
Bibliographic Details
Published inSystems & control letters Vol. 100; pp. 14 - 20
Main Author Vamvoudakis, Kyriakos G.
Format Journal Article
LanguageEnglish
Published Elsevier B.V 01.02.2017
Subjects
Actor/critic approximators
Optimal control
Q-learning
Uncertain systems
Uncertain systems
Actor/critic approximators
Q-learning
Optimal control
Online AccessGet full text

Cover

More Information
Summary:In this paper we propose an online Q-learning algorithm to solve the infinite-horizon optimal control problem of a linear time invariant system with completely uncertain/unknown dynamics. We first formulate the Q-function by using the Hamiltonian and the optimal cost. An integral reinforcement learning approach is used to develop an actor/critic approximator structure to estimate the parameters of the Q-function online while also guaranteeing closed-loop asymptotic stability and convergence to the optimal solution.
ISSN:0167-6911
1872-7956
DOI:10.1016/j.sysconle.2016.12.003