Variational Bayesian Inference for Parametric and Nonparametric Regression With Missing Data

• Published in: Journal of the American Statistical Association Vol. 106; no. 495; pp. 959 - 971
• Main Authors: Faes, C., Ormerod, J. T., Wand, M. P.
• Format: Journal Article
• Language: English
• Published: Alexandria, VA Taylor & Francis 01.09.2011; American Statistical Association; Taylor & Francis Ltd
• Subjects: Applications; Approximation; Bayesian analysis; Bayesian inference; Bayesian method; Computational methods; Directed acyclic graphs; Exact sciences and technology; General topics; Incomplete data; Inference; Insurance, economics, finance; Linear regression; Mathematics; Mean field approximation; Missing data; Modeling; Monte Carlo simulation; Nonparametric inference; Parameter estimation; Parametric inference; Parametric models; Penalized splines; Probability and statistics; Probability calculus; Regression analysis; Sciences and techniques of general use; Statistics; Stochastic models; Theory and Methods; Variational approximation; Wands; Statistical distribution; Directed acyclic graphs; Non parametric estimation; Probability distribution; Penalized splines; Economic sciences; Penalty method; Spline approximation; Numerical approximation; Parametric method; Approximation theory; Incomplete information; Deterministic approach; Acyclic graph; Bayes estimation; Monte Carlo method; Data analysis; Mean field approximation; Nonparametric regression; Probability; Incomplete data; Statistical estimation; Regression analysis; Hierarchized structure; Statistical method; Missing data; Regression model; Hierarchical model; Variational approximation; Econometrics; Parametric inference; Directed graph
• ISSN: 0162-1459; 1537-274X
• DOI: 10.1198/jasa.2011.tm10301

Bayesian hierarchical models are attractive structures for conducting regression analyses when the data are subject to missingness. However, the requisite probability calculus is challenging and Monte Carlo methods typically are employed. We develop an alternative approach based on deterministic variational Bayes approximations. Both parametric and nonparametric regression are considered. Attention is restricted to the more challenging case of missing predictor data. We demonstrate that variational Bayes can achieve good accuracy, but with considerably less computational overhead. The main ramification is fast approximate Bayesian inference in parametric and nonparametric regression models with missing data. Supplemental materials accompany the online version of this article.