Optimal stopping with a capacity constraint: Generalizing Shepp’s urn scheme

We formulate an optimal stopping problem for a variant of Shepp’s urn model in which it is possible to sample more than one item at each stage. Using a Markov decision process model, we establish monotonicity of the optimal value function and show that the optimal policy is a monotone threshold poli...

Full description

Saved in:
Bibliographic Details
Published inOperations research letters Vol. 47; no. 4; pp. 311 - 316
Main Authors Dehghanian, Amin, Kharoufeh, Jeffrey P.
Format Journal Article
LanguageEnglish
Published Elsevier B.V 01.07.2019
Subjects
Markov decision process
Optimal stopping
Sampling policy
Urn models
Urn models
Markov decision process
Optimal stopping
Sampling policy
Online AccessGet full text

Cover

More Information
Summary:We formulate an optimal stopping problem for a variant of Shepp’s urn model in which it is possible to sample more than one item at each stage. Using a Markov decision process model, we establish monotonicity of the optimal value function and show that the optimal policy is a monotone threshold policy that prescribes either not sampling, or sampling the maximum number of items permitted. A special case exhibits convexity and submodularity, but these properties do not hold in general.
ISSN:0167-6377
1872-7468
DOI:10.1016/j.orl.2019.04.006