On the Sample Complexity of the Linear Quadratic Regulator

• Published in: Foundations of computational mathematics Vol. 20; no. 4; pp. 633 - 679
• Main Authors: Dean, Sarah, Mania, Horia, Matni, Nikolai, Recht, Benjamin, Tu, Stephen
• Format: Journal Article
• Language: English
• Published: New York Springer US 01.08.2020; Springer; Springer Nature B.V
• Subjects: Analysis; Applications of Mathematics; Computational geometry; Computer Science; Control systems design; Controllers; Convexity; Cost control; Design optimization; Economics; Errors; Linear and Multilinear Algebras; Linear quadratic regulator; Math Applications in Computer Science; Mathematics; Mathematics and Statistics; Matrix Theory; Numerical Analysis; Optimal control; Robust control; Synthesis; Uncertainty; Robust control; System level synthesis; 93D21; Optimal control; 93E35; Reinforcement learning; 93E12; 93D09; 49N05; Statistical learning theory; System identification
• ISSN: 1615-3375; 1615-3383
• DOI: 10.1007/s10208-019-09426-y

This paper addresses the optimal control problem known as the linear quadratic regulator in the case when the dynamics are unknown. We propose a multistage procedure, called Coarse-ID control , that estimates a model from a few experimental trials, estimates the error in that model with respect to the truth, and then designs a controller using both the model and uncertainty estimate. Our technique uses contemporary tools from random matrix theory to bound the error in the estimation procedure. We also employ a recently developed approach to control synthesis called System Level Synthesis that enables robust control design by solving a quasi-convex optimization problem. We provide end-to-end bounds on the relative error in control cost that are optimal in the number of parameters and that highlight salient properties of the system to be controlled such as closed-loop sensitivity and optimal control magnitude. We show experimentally that the Coarse-ID approach enables efficient computation of a stabilizing controller in regimes where simple control schemes that do not take the model uncertainty into account fail to stabilize the true system.