Constrained Stochastic LQC: A Tractable Approach

• Published in: IEEE transactions on automatic control Vol. 52; no. 10; pp. 1826 - 1841
• Main Authors: Bertsimas, D., Brown, D.B.
• Format: Journal Article
• Language: English
• Published: New York, NY IEEE 01.10.2007; Institute of Electrical and Electronics Engineers; The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
• Subjects: Algorithms; Applied sciences; Approximation; Computer science; control theory; systems; Constraint optimization; Control systems; Control theory. Systems; Control with constraints; Cost function; Dynamic programming; Exact sciences and technology; Linear systems; linear-quadratic control; Mathematical analysis; Optimization; Riccati equations; Robust control; robust optimization; Scalars; semidefinite optimization; State-space methods; Stochastic processes; Stochastic systems; Stochasticity; Studies; Control with constraints; linear-quadratic control; Probabilistic approach; robust optimization; Semi definite programming; State space method; Distributed system; Quadratic programming; Convex programming; Optimization; Closed feedback; Control constraint; LQ control; Linear system; Matrix calculus; Stochastic control; Dynamic programming; Riccati equation; semidefinite optimization; State constraint
• ISSN: 0018-9286; 1558-2523
• DOI: 10.1109/TAC.2007.906182

Despite the celebrated success of dynamic programming for optimizing quadratic cost functions over linear systems, such an approach is limited by its inability to tractably deal with even simple constraints. In this paper, we present an alternative approach based on results from robust optimization to solve the stochastic linear-quadratic control (SLQC) problem. In the unconstrained case, the problem may be formulated as a semidefinite optimization problem (SDP). We show that we can reduce this SDP to optimization of a convex function over a scalar variable followed by matrix multiplication in the current state, thus yielding an approach that is amenable to closed-loop control and analogous to the Riccati equation in our framework. We also consider a tight, second-order cone (SOCP) approximation to the SDP that can be solved much more efficiently when the problem has additional constraints. Both the SDP and SOCP are tractable in the presence of control and state space constraints; moreover, compared to the Riccati approach, they provide much greater control over the stochastic behavior of the cost function when the noise in the system is distributed normally.