High-Dimensional Copula Variational Approximation Through Transformation

• Published in: Journal of computational and graphical statistics Vol. 29; no. 4; pp. 729 - 743
• Main Authors: Smith, Michael Stanley, Loaiza-Maya, Rubén, Nott, David J.
• Format: Journal Article
• Language: English
• Published: Alexandria Taylor & Francis 01.10.2020; Taylor & Francis Ltd
• Subjects: Accuracy; Approximation; Bayesian analysis; Computational efficiency; Computing costs; Datasets; Factor variational approximation; Implicit copula; Inverse G&H transformation; Normal distribution; Skew-normal copula; Skewed distributions; Statistical inference; Transformations (mathematics); Variational methods; Yeo-Johnson transformation
• ISSN: 1061-8600; 1537-2715
• DOI: 10.1080/10618600.2020.1740097

Variational methods are attractive for computing Bayesian inference when exact inference is impractical. They approximate a target distribution-either the posterior or an augmented posterior-using a simpler distribution that is selected to balance accuracy with computational feasibility. Here, we approximate an element-wise parametric transformation of the target distribution as multivariate Gaussian or skew-normal. Approximations of this kind are implicit copula models for the original parameters, with a Gaussian or skew-normal copula function and flexible parametric margins. A key observation is that their adoption can improve the accuracy of variational inference in high dimensions at limited or no additional computational cost. We consider the Yeo-Johnson and inverse G&H transformations, along with sparse factor structures for the scale matrix of the Gaussian or skew-normal. We also show how to implement efficient reparameterization gradient methods for these copula-based approximations. The efficacy of the approach is illustrated by computing posterior inference for three different models using six real datasets. In each case, we show that our proposed copula model distributions are more accurate variational approximations than Gaussian or skew-normal distributions, but at only a minor or no increase in computational cost. Supplementary materials comprising an online appendix, MATLAB code to implement the method, and the datasets employed, are available online.