The existence of a minimum pair of state and policy for Markov decision processes under the hypothesis of Doeblin

This paper studies the average-cost Markov decision process with compact state and action spaces and bounded lower semicontinuous cost functions. Following the idea of Borkar's excellent papers [SIAMJ. Control Optim., 21 (1983), pp. 652-666; 22 (1984), pp. 965-978], the general case where irred...

Full description

Saved in:
Bibliographic Details
Published inSIAM journal on control and optimization Vol. 27; no. 2; pp. 296 - 307
Main Author KURANO, M
Format Journal Article
LanguageEnglish
Published Philadelphia, PA Society for Industrial and Applied Mathematics 01.03.1989
Subjects
Applied sciences
Decision theory. Utility theory
Exact sciences and technology
Hypotheses
Markov analysis
Operational research and scientific management
Operational research. Management science
Discontinuity
Lower bound
Doeblin condition
Markov decision
Decision theory
Average cost
Optimal policy
Cost function
Online AccessGet full text

Cover

More Information
Summary:This paper studies the average-cost Markov decision process with compact state and action spaces and bounded lower semicontinuous cost functions. Following the idea of Borkar's excellent papers [SIAMJ. Control Optim., 21 (1983), pp. 652-666; 22 (1984), pp. 965-978], the general case where irreducibility is not assumed is considered under the hypothesis of Doeblin and the existence of a minimum pair of state and policy, which attains the infimum of the average expected cost over all initial states and policies, is established. Further, it is proved that under additional weak conditions there exists an optimal stationary policy in the usual sense.
ISSN:0363-0129
1095-7138
DOI:10.1137/0327016