Meta-Analyses as a Multi-Level Model

• Published in: Organizational research methods Vol. 24; no. 2; pp. 389 - 411
• Main Authors: Gooty, Janaki, Banks, George C., Loignon, Andrew C., Tonidandel, Scott, Williams, Courtney E.
• Format: Journal Article
• Language: English
• Published: Los Angeles, CA SAGE Publications 01.04.2021; SAGE PUBLICATIONS, INC
• Subjects: Medical research; Meta-analysis; Monte Carlo simulation; Psychology; Software; multilevel analysis; meta-analysis; dependency
• ISSN: 1094-4281; 1552-7425
• DOI: 10.1177/1094428119857471

Meta-analyses are well known and widely implemented in almost every domain of research in management as well as the social, medical, and behavioral sciences. While this technique is useful for determining validity coefficients (i.e., effect sizes), meta-analyses are predicated on the assumption of independence of primary effect sizes, which might be routinely violated in the organizational sciences. Here, we discuss the implications of violating the independence assumption and demonstrate how meta-analysis could be cast as a multilevel, variance known (Vknown) model to account for such dependency in primary studies’ effect sizes. We illustrate such techniques for meta-analytic data via the HLM 7.0 software as it remains the most widely used multilevel analyses software in management. In so doing, we draw on examples in educational psychology (where such techniques were first developed), organizational sciences, and a Monte Carlo simulation (Appendix). We conclude with a discussion of implications, caveats, and future extensions. Our Appendix details features of a newly developed application that is free (based on R), user-friendly, and provides an alternative to the HLM program.