Dynamic programming equations for constrained stochastic control

• Published in: Proceedings of the 2002 American Control Conference (IEEE Cat. No.CH37301) Vol. 3; pp. 2014 - 2022 vol.3
• Main Authors: Chen, R.C., Blankenship, G.L.
• Format: Conference Proceeding
• Language: English
• Published: Piscataway NJ IEEE 2002
• Subjects: Applied sciences; Computer science; control theory; systems; Constraint theory; Control theory. Systems; Costs; Dynamic programming; Dynamic scheduling; Equations; Exact sciences and technology; Infinite horizon; Minimax techniques; Optimal control; Stochastic processes; Strain control; Costs; Discount; Existence theorem; Dynamic theory; Control system; Numerical method; Uniqueness theorem; Constrained optimization; Stochastic programming; Control constraint; Fixed point theorem; Minimax problem; Minimax method; Stochastic control; Infinite horizon; Dynamic programming; Stochastic equation
• ISBN: 0780372980; 9780780372986
• ISSN: 0743-1619
• DOI: 10.1109/ACC.2002.1023931

It is demonstrated that the dynamic programming approach provides a simple and versatile means for analyzing constrained stochastic control problems. Specifically, three such problems are analyzed using this approach. The problems analyzed are the problem of minimizing a discounted cost infinite horizon expectation objective subject to an identically structured constraint, the problem of minimizing a discounted cost infinite horizon minimax objective subject to a discounted expectation constraint, and the problem of minimizing a discounted expected cost objective subject to a minimax constraint. Using a dynamic programming approach, optimality equations are obtained for these problems. The optimality equations derived for the first two problems are apparently novel. Existence and uniqueness of solutions to the dynamic programming equations for the discounted cost infinite horizon problems are explicitly shown using the Banach fixed point theorem. Thus, the paper demonstrates that the dynamic programming theory for unconstrained stochastic control problems can be extended in a direct way to constrained stochastic control problems. The theory developed is illustrated by numerically solving the constrained stochastic control dynamic programming equations derived for simple example problems.