Discrete-time LQ-optimal control problems for infinite Markov jump parameter systems

• Published in: IEEE transactions on automatic control Vol. 40; no. 12; pp. 2076 - 2088
• Main Authors: Costa, O.L.V., Fragoso, M.D.
• Format: Journal Article
• Language: English
• Published: New York, NY IEEE 01.12.1995; Institute of Electrical and Electronics Engineers
• Subjects: Additive noise; Applied sciences; Brazil Council; Computer science; control theory; systems; Control systems; Control theory. Systems; Cost function; Equations; Exact sciences and technology; Integrated circuit modeling; Linear systems; Nonlinear control systems; Optimal control; Stochastic processes; Markov process; LQ control; Linear system; Optimal control; Average cost; Algebraic equation; Jump process; Discrete system; Quadratic cost; Infinite horizon; Riccati equation
• ISSN: 0018-9286
• DOI: 10.1109/9.478328

Optimal control problems for discrete-time linear systems subject to Markovian jumps in the parameters are considered for the case in which the Markov chain takes values in a countably infinite set. Two situations are considered: the noiseless case and the case in which an additive noise is appended to the model. The solution for these problems relies, in part, on the study of a countably infinite set of coupled algebraic Riccati equations (ICARE). Conditions for existence and uniqueness of a positive semidefinite solution to the ICARE are obtained via the extended concepts of stochastic stabilizability (SS) and stochastic detectability (SD), which turn out to be equivalent to the spectral radius of certain infinite dimensional linear operators in a Banach space being less than one. For the long-run average cost, SS and SD guarantee existence and uniqueness of a stationary measure and consequently existence of an optimal stationary control policy. Furthermore, an extension of a Lyapunov equation result is derived for the countably infinite Markov state-space case.