A generalized iterative LQG method for locally-optimal feedback control of constrained nonlinear stochastic systems

• Published in: 2005 American Control Conference pp. 300 - 306 vol. 1
• Main Authors: Todorov, E., Weiwei Li
• Format: Conference Proceeding
• Language: English
• Published: IEEE 2005
• Subjects: Control systems; Convergence; Costs; Feedback control; Iterative methods; Linear feedback control systems; Mathematical model; Nonlinear control systems; Stochastic processes; Stochastic systems
• ISBN: 0780390989; 9780780390980; 9780780390997; 0780390997
• ISSN: 0743-1619
• DOI: 10.1109/ACC.2005.1469949

We present an iterative linear-quadratic-Gaussian method for locally-optimal feedback control of nonlinear stochastic systems subject to control constraints. Previously, similar methods have been restricted to deterministic unconstrained problems with quadratic costs. The new method constructs an affine feedback control law, obtained by minimizing a novel quadratic approximation to the optimal cost-to-go function. Global convergence is guaranteed through a Levenberg-Marquardt method; convergence in the vicinity of a local minimum is quadratic. Performance is illustrated on a limited-torque inverted pendulum problem, as well as a complex biomechanical control problem involving a stochastic model of the human arm, with 10 state dimensions and 6 muscle actuators. A Matlab implementation of the new algorithm is availabe at www.cogsci.ucsd.edu//spl sim/todorov.