Selecting Shrinkage Parameters for Effect Estimation

• Published in: American journal of epidemiology Vol. 187; no. 2; pp. 358 - 365
• Main Authors: Keller, Joshua P, Rice, Kenneth M
• Format: Journal Article
• Language: English
• Published: Oxford University Press 01.02.2018
• Subjects: confounding factors (epidemiology); model selection; shrinkage estimators; LASSO
• DOI: 10.1093/aje/kwx225
• ISSN: 0002-9262

Abstract We present a method for improving estimation in linear regression models in samples of moderate size, using shrinkage techniques. Our work connects the theory of causal inference, which describes how variable adjustment should be performed with large samples, with shrinkage estimators such as ridge regression and the least absolute shrinkage and selection operator (LASSO), which can perform better in sample sizes seen in epidemiologic practice. Shrinkage methods reduce mean squared error by trading off some amount of bias for a reduction in variance. However, when inference is the goal, there are no standard methods for choosing the penalty “tuning” parameters that govern these tradeoffs. We propose selecting the penalty parameters for these shrinkage estimators by minimizing bias and variance in future similar data sets drawn from the posterior predictive distribution. Our method provides both the point estimate of interest and corresponding standard error estimates. Through simulations, we demonstrate that it can achieve better mean squared error than using cross-validation for penalty parameter selection. We apply our method to a cross-sectional analysis of the association between smoking and carotid intima-media thickness in the Multi-Ethnic Study of Atherosclerosis (multiple US locations, 2000–2002) and compare it with similar analyses of these data.