A three-term recurrence relation for accurate evaluation of transition probabilities of the simple birth-and-death process

• Published in: BIT Vol. 61; no. 2; pp. 561 - 585
• Main Authors: Pessia, Alberto, Tang, Jing
• Format: Journal Article
• Language: English
• Published: Dordrecht Springer Netherlands 01.06.2021; Springer Nature B.V
• Subjects: Computational Mathematics and Numerical Analysis; Hypergeometric functions; Mathematics; Mathematics and Statistics; Maximum likelihood estimates; Numeric Computing; Numerical methods; Stochastic models; Transition probabilities; Transition probabilities; Secondary 60J80; Maximum likelihood estimation; Primary 65Q30; Birth-death process; 62M05; Hypergeometric function; 62F10; Three-term recurrence relation; 33C05
• ISSN: 0006-3835; 1572-9125
• DOI: 10.1007/s10543-020-00836-x

The simple (linear) birth-and-death process is a widely used stochastic model for describing the dynamics of a population. When the process is observed discretely over time, despite the large amount of literature on the subject, little is known about formal estimator properties. Here we will show that its application to observed data is further complicated by the fact that numerical evaluation of the well-known transition probability is an ill-conditioned problem. To overcome this difficulty we will rewrite the transition probability in terms of a Gaussian hypergeometric function and subsequently obtain a three-term recurrence relation for its accurate evaluation. We will also study the properties of the hypergeometric function as a solution to the three-term recurrence relation. We will then provide formulas for the gradient and Hessian of the log-likelihood function and conclude the article by applying our methods for numerically computing maximum likelihood estimates in both simulated and real dataset.