On Optimizing the Conditional Value-at-Risk of a Maximum Cost for Risk-Averse Safety Analysis

• Published in: IEEE transactions on automatic control Vol. 68; no. 6; pp. 3720 - 3727
• Main Authors: Chapman, Margaret P., Faus, Michael, Smith, Kevin M.
• Format: Journal Article
• Language: English
• Published: New York IEEE 01.06.2023; The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
• Subjects: Aerospace electronics; Combined sewer overflows; Conditional Value-at-Risk (CVaR); Cost analysis; Costs; Markov decision processes (MDPs); Markov processes; Optimal control; Optimization; Probability theory; Process control; Random variables; Risk analysis; risk-averse optimal control; Safety; safety analysis; Stochastic systems
• ISSN: 0018-9286; 1558-2523
• DOI: 10.1109/TAC.2022.3195381

The popularity of Conditional Value-at-Risk (CVaR), a risk functional from finance, has been growing in the control systems community due to its intuitive interpretation and axiomatic foundation. We consider a nonstandard optimal control problem in which the goal is to minimize the CVaR of a maximum random cost subject to a Borel-space Markov decision process. The objective represents the maximum departure from a desired operating region averaged over a given fraction of the worst cases. This problem provides a safety criterion for a stochastic system that is informed by both the probability and severity of the potential consequences of the system's behavior. In contrast, existing safety analysis frameworks apply stagewise risk constraints or assess the probability of constraint violation without quantifying the potential severity of the violation. To the best of our knowledge, the problem of interest has not been solved. To solve the problem, we propose and study a family of stochastic dynamic programs on an augmented state space. We prove that the optimal CVaR of a maximum random cost enjoys an equivalent representation in terms of the solutions to these dynamic programs under appropriate assumptions. For each dynamic program, we show the existence of an optimal policy that depends on the dynamics of an augmented state under the assumptions. In a numerical example, we illustrate how our safety analysis framework is useful for assessing the severity of combined sewer overflows under precipitation uncertainty.