ASYMPTOTIC VARIANCE OF PASSAGE TIME ESTIMATORS IN MARKOV CHAINS

We consider the problem of estimating passage times in stochastic simulations of Markov chains. Two types of estimator are considered for this purpose: the “simple” and the “overlapping” estimator; they are compared in terms of their asymptotic variance. The analysis is based on the regenerative str...

Full description

Saved in:
Bibliographic Details
Published inProbability in the engineering and informational sciences Vol. 21; no. 2; pp. 217 - 234
Main Author Zazanis, Michael A.
Format Journal Article
LanguageEnglish
Published New York, USA Cambridge University Press 01.04.2007
Subjects
Online AccessGet full text

Cover

Loading…
More Information
Summary:We consider the problem of estimating passage times in stochastic simulations of Markov chains. Two types of estimator are considered for this purpose: the “simple” and the “overlapping” estimator; they are compared in terms of their asymptotic variance. The analysis is based on the regenerative structure of the process and it is shown that when estimating the mean passage time, the simple estimator is always asymptotically superior. However, when the object is to estimate the expectation of a nonlinear function of the passage time, such as the probability that the passage time exceeds a given threshold, then it is shown that the overlapping estimator can be superior in some cases. Related results in the Reinforcement Learning literature are discussed.
Bibliography:ark:/67375/6GQ-JVRKZ1H8-K
istex:231457B7941F909B930AEF2A19C40A479E340536
PII:S0269964807070143
ISSN:0269-9648
1469-8951
DOI:10.1017/S0269964807070143