Characterization of the optimal average cost in Markov decision chains driven by a risk-seeking controller

• Published in: Journal of applied probability Vol. 61; no. 1; pp. 340 - 367
• Main Authors: Cavazos-Cadena, Rolando, Cruz-Suárez, Hugo, Montes-de-Oca, Raúl
• Format: Journal Article
• Language: English
• Published: Cambridge, UK Cambridge University Press 01.03.2024
• Subjects: Cost function; Dynamical systems; Expected values; Markov chains; Mathematical functions; Original Article; Random variables; Risk communication; Utility functions; Risk-lover controller; 93E20; 93C55; truncated cost function; optimality inequality; optional sampling theorem; extended Collatz–Wielandt formula; exponential utility
• ISSN: 0021-9002; 1475-6072
• DOI: 10.1017/jpr.2023.40

This work concerns Markov decision chains on a denumerable state space endowed with a bounded cost function. The performance of a control policy is assessed by a long-run average criterion as measured by a risk-seeking decision maker with constant risk-sensitivity. Besides standard continuity–compactness conditions, the framework of the paper is determined by the following conditions: (i) the state process is communicating under each stationary policy, and (ii) the simultaneous Doeblin condition holds. Within this framework it is shown that (i) the optimal superior and inferior limit average value functions coincide and are constant, and (ii) the optimal average cost is characterized via an extended version of the Collatz–Wielandt formula in the theory of positive matrices.