ASYMPTOTIC VARIANCE OF PASSAGE TIME ESTIMATORS IN MARKOV CHAINS
We consider the problem of estimating passage times in stochastic simulations of Markov chains. Two types of estimator are considered for this purpose: the “simple” and the “overlapping” estimator; they are compared in terms of their asymptotic variance. The analysis is based on the regenerative str...
Saved in:
Published in | Probability in the engineering and informational sciences Vol. 21; no. 2; pp. 217 - 234 |
---|---|
Main Author | |
Format | Journal Article |
Language | English |
Published |
New York, USA
Cambridge University Press
01.04.2007
|
Subjects | |
Online Access | Get full text |
Cover
Loading…
Summary: | We consider the problem of estimating passage times in stochastic
simulations of Markov chains. Two types of estimator are considered for
this purpose: the “simple” and the “overlapping”
estimator; they are compared in terms of their asymptotic variance. The
analysis is based on the regenerative structure of the process and it is
shown that when estimating the mean passage time, the simple
estimator is always asymptotically superior. However, when the object is
to estimate the expectation of a nonlinear function of the passage time,
such as the probability that the passage time exceeds a given
threshold, then it is shown that the overlapping estimator can be
superior in some cases. Related results in the Reinforcement Learning
literature are discussed. |
---|---|
Bibliography: | ark:/67375/6GQ-JVRKZ1H8-K istex:231457B7941F909B930AEF2A19C40A479E340536 PII:S0269964807070143 |
ISSN: | 0269-9648 1469-8951 |
DOI: | 10.1017/S0269964807070143 |