AN EFFICIENT FINITE DIFFERENCE METHOD FOR PARAMETER SENSITIVITIES OF CONTINUOUS TIME MARKOV CHAINS

• Published in: SIAM journal on numerical analysis Vol. 50; no. 5; pp. 2237 - 2258
• Main Author: ANDERSON, DAVID F.
• Format: Journal Article
• Language: English
• Published: Philadelphia Society for Industrial and Applied Mathematics 01.01.2012
• Subjects: Algorithms; Applied mathematics; Estimation methods; Estimators; Finite difference method; Finite difference methods; Joining; Markov analysis; Markov chains; Mathematical analysis; Mathematical functions; Mathematical models; Methods; Molecules; Parametric models; Poisson process; Random variables; Stochastic models; Variance
• ISSN: 0036-1429; 1095-7170
• DOI: 10.1137/110849079

We present an efficient finite difference method for the computation of parameter sensitivities that is applicable to a wide class of continuous time Markov chain models. The estimator for the method is constructed by coupling the perturbed and nominal processes in a natural manner, and the analysis proceeds by utilizing a martingale representation for the coupled processes. The variance of the resulting estimator is shown to be an order of magnitude lower due to the coupling. We conclude that the proposed method produces an estimator with a lower variance than other methods, including the use of common random numbers, in most situations. Often the variance reduction is substantial. The method is no harder to implement than any standard continuous time Markov chain algorithm, such as "Gillespie's algorithm." The motivating class of models, and the source of our examples, are the stochastic chemical kinetic models commonly used in the biosciences, though other natural application areas include population processes and queuing networks.