Mostly harmless simulations? Using Monte Carlo studies for estimator selection

Summary We consider two recent suggestions for how to perform an empirically motivated Monte Carlo study to help select a treatment effect estimator under unconfoundedness. We show theoretically that neither is likely to be informative except under restrictive conditions that are unlikely to be sati...

Full description

Saved in:
Bibliographic Details
Published inJournal of applied econometrics (Chichester, England) Vol. 34; no. 6; pp. 893 - 910
Main Authors Advani, Arun, Kitagawa, Toru, Słoczyński, Tymon
Format Journal Article
LanguageEnglish
Published Chichester Wiley Periodicals Inc 01.09.2019
Subjects
Bias
Computer simulation
Econometrics
Economic models
Estimators
Monte Carlo simulation
Robustness
Online AccessGet full text

Cover

More Information
Summary:Summary We consider two recent suggestions for how to perform an empirically motivated Monte Carlo study to help select a treatment effect estimator under unconfoundedness. We show theoretically that neither is likely to be informative except under restrictive conditions that are unlikely to be satisfied in many contexts. To test empirical relevance, we also apply the approaches to a real‐world setting where estimator performance is known. Both approaches are worse than random at selecting estimators that minimize absolute bias. They are better when selecting estimators that minimize mean squared error. However, using a simple bootstrap is at least as good and often better. For now, researchers would be best advised to use a range of estimators and compare estimates for robustness.
ISSN:0883-7252
1099-1255
DOI:10.1002/jae.2724