Linear programming formulation for non-stationary, finite-horizon Markov decision process models

Linear programming (LP) formulations are often employed to solve stationary, infinite-horizon Markov decision process (MDP) models. We present an LP approach to solving non-stationary, finite-horizon MDP models that can potentially overcome the computational challenges of standard MDP solution proce...

Full description

Saved in:
Bibliographic Details
Published inOperations research letters Vol. 45; no. 6; pp. 570 - 574
Main Authors Bhattacharya, Arnab, Kharoufeh, Jeffrey P.
Format Journal Article
LanguageEnglish
Published Elsevier B.V 01.11.2017
Subjects
Linear programming
Non-stationary MDP model
Linear programming
Non-stationary MDP model
Online AccessGet full text

Cover

More Information
Summary:Linear programming (LP) formulations are often employed to solve stationary, infinite-horizon Markov decision process (MDP) models. We present an LP approach to solving non-stationary, finite-horizon MDP models that can potentially overcome the computational challenges of standard MDP solution procedures. Specifically, we establish the existence of an LP formulation for risk-neutral MDP models whose states and transition probabilities are temporally heterogeneous. This formulation can be recast as an approximate linear programming formulation with significantly fewer decision variables.
ISSN:0167-6377
1872-7468
DOI:10.1016/j.orl.2017.09.001