Nonparametric Estimation of the Transition Distribution Function of a Markov Process

• Published in: The Annals of mathematical statistics Vol. 40; no. 4; pp. 1386 - 1400
• Main Author: Roussas, George G.
• Format: Journal Article
• Language: English
• Published: Institute of Mathematical Statistics 01.08.1969; The Institute of Mathematical Statistics
• Subjects: Distribution functions; Markov processes; Random variables
• ISSN: 0003-4851; 2168-8990
• DOI: 10.1214/aoms/1177697510

In [8] the problem of nonparametric estimation in Markov processes has been considered, and estimates of the initial, two-dimensional joint, and transition densities of the process, satisfying a number of optimal properties, have been obtained. In the present paper, under the same nonparametric setup, the attention is centered primarily on the transition distribution function of the process. It will be assumed that the underlying Markov process, defined on a probability space (Ω, a, P) and taking values in the real line R, is (strictly) stationary, and has initial, two-dimensional joint, and transition densities p(·), q(·, ·), and t(· ∣ x), x ε R, respectively, relative to the appropriate Lebesgue measures. Let K be a probability density. On the basis of the first n + 1 random variables Xj, j = 1, 2, ⋯, n + 1 of the process, we define the random variables pn(x), x ε R, and qn(y), y ε R × R (by suppressing the random element ω) by the following relations \begin{equation*} \tag{1.1} p_n(x) = (nh)^{-1} \sum^n_{j=1} K((x - X_j)h^{-1})\end{equation*} \begin{equation*} \tag{1.2} q_n(y) = q_n(x, x') = (nh)^{-1} \sum^n_{j=1} K((x - X_j)h^{-\frac{1}{2}})K((x' - X_{j+1})h^{-\frac{1}{2}}),\end{equation*} where h = h(n) is a sequence of positive constants satisfying also some additional conditions. We further set \begin{equation*} \tag{1.3} t_n(x' \mid x) = q_n(x, x')/p_n(x).\end{equation*} Next, by means of pn(x) and qn(x, x'), define the random variables$F_n(x) = \int^x_{-\infty} p_n(z) dz, \quad G_n(z \mid x) = \int^z_{-\infty} t_n(dx' \mid x).$We finally let F(·) and G(· ∣ x), x ε R, be the initial and transition distribution functions of the process. Under suitable conditions on the function K, the sequence {h}, and the process, the main results of this paper are the following. The distribution function Fn(x), as an estimate of F(x), obeys the Glivenko-Cantelli theorem. This is Theorem 3.1. Turning now to the estimate Gn(· | x) of G(· | x), we have been able to establish in Theorem 3.2 that$\sup \{|G_n(z | x) - G(z | x)a|; z \varepsilon R\}$converges to zero, as n → ∞, but in the probability sense. This is true for all x ε R. In Section 4 it is assumed that the rth absolute moment of X1exists (for some r = 1, 2, ⋯), and the problem is that of gaining further information about G(· ∣ x), by estimating its kth moment, to be denoted by m(k; x), for k = 1, 2, ⋯, r. By letting the rather simple expression mn(k; x) = (nh)-1pn -1(x) ∑n j=1Xk j+1K((x - Xj)h-1) stand for an estimate of m(k; x), it is shown that, as n → ∞, mn(k; x) → m(k; x) in probability, for k = 1, 2, ⋯, r and x ε R. This is the content of Theorem 4.1. Finally in Section 5 we look into the problem of estimating the quantiles of G(· ∣ x), and in connection with this, two results are derived. For some p in the interval (0, 1), it is assumed that the pth quantile, ξ(p, x), of G(· ∣ x) is unique. By defining ξn(p, x) as the smallest root of the equation Gn(z ∣ x) = p and using it as an estimate of ξ(p, x), it is proved that, as n → ∞,$\xi_n(p, x) \rightarrow \xi(p, x)\quad \text{in probability (Theorem 5.1)},$and$(nh)^{\frac{1}{2}} \lbrack \xi_n(p, x) - \xi(p, x) \rbrack \rightarrow N(0, \tau^2(\xi, x))\quad \text{in law}.$This is Theorem 5.2, and the variance τ2(ξ, x) is explicitly given in that theorem.