Model-free optimal control of discrete-time systems with additive and multiplicative noises

• Published in: Automatica (Oxford) Vol. 147; p. 110685
• Main Authors: Lai, Jing, Xiong, Junlin, Shu, Zhan
• Format: Journal Article
• Language: English
• Published: Elsevier Ltd 01.01.2023
• Subjects: Additive and multiplicative noises; Model-free; Reinforcement learning; Stochastic linear quadratic regulator; Stochastic linear quadratic regulator; Model-free; Additive and multiplicative noises; Reinforcement learning
• ISSN: 0005-1098; 1873-2836
• DOI: 10.1016/j.automatica.2022.110685

This paper investigates the optimal control problem for a class of discrete-time stochastic systems subject to additive and multiplicative noises. An algebraic Riccati equation is established which gives the form of the solution to the problem. To obtain the optimal control gain iteratively, an offline policy iteration is presented with convergence proof. A model-free reinforcement learning algorithm is proposed to learn the optimal admissible control policy using the system states and inputs without resorting to the system matrices. It is proven that the estimation error of the kernel matrix is bounded and the iterative control gain is admissible. Compared with the existing work, this paper considers the model-free controller learning problem for stochastic systems suffering from both additive and multiplicative noises using reinforcement learning. The proposed algorithm is illustrated through a numerical example, which shows that our algorithm outperforms other policy iteration algorithms.