Covariate Balance in Bayesian Propensity Score Approaches for Observational Studies

• Published in: Journal of research on educational effectiveness Vol. 8; no. 2; pp. 280 - 302
• Main Authors: Chen, Jianshen, Kaplan, David
• Format: Journal Article
• Language: English
• Published: Philadelphia Routledge 03.04.2015; Taylor & Francis Ltd
• Subjects: Accuracy; Bayesian analysis; Bayesian Statistics; Case Studies; Comparative Analysis; Correlation; covariate balance; Kindergarten; Longitudinal Studies; Observation; Propensity scores; Reading Achievement; School Schedules; Scores; Simulation; Weighted Scores
• ISSN: 1934-5747; 1934-5739
• DOI: 10.1080/19345747.2014.911396

Bayesian alternatives to frequentist propensity score approaches have recently been proposed. However, few studies have investigated their covariate balancing properties. This article compares a recently developed two-step Bayesian propensity score approach to the frequentist approach with respect to covariate balance. The effects of different priors on covariate balance are evaluated and the differences between frequentist and Bayesian covariate balance are discussed. Results of the case study reveal that both the Bayesian and frequentist propensity score approaches achieve good covariate balance. The frequentist propensity score approach performs slightly better on covariate balance for stratification and weighting methods, whereas the two-step Bayesian approach offers slightly better covariate balance in the optimal full matching method. Results of a comprehensive simulation study reveal that accuracy and precision of prior information on propensity score model parameters do not greatly influence balance performance. Results of the simulation study also show that overall, the optimal full matching method provides the best covariate balance and treatment effect estimates compared to the stratification and weighting methods. A unique feature of covariate balance within Bayesian propensity score analysis is that we can obtain a distribution of balance indices in addition to the point estimates so that the variation in balance indices can be naturally captured to assist in covariate balance checking.