Split‐sample reliability estimation in health care quality measurement: Once is not enough

• Published in: Health services research Vol. 59; no. 4; pp. e14310 - n/a
• Main Authors: Nieser, Kenneth J., Harris, Alex H. S.
• Format: Journal Article
• Language: English
• Published: Oxford, UK Blackwell Publishing Ltd 01.08.2024
• Subjects: Data collection; Estimates; Extraction; Health care; health care quality; Measurement; performance measurement; Quality assessment; Quality of care; Reliability; Simulation; split‐half reliability; split‐sample reliability; health care quality; performance measurement; split‐half reliability; split‐sample reliability
• ISSN: 0017-9124; 1475-6773; 1475-6773
• DOI: 10.1111/1475-6773.14310

Objective To examine the sensitivity of split‐sample reliability estimates to the random split of the data and propose alternative methods for improving the stability of the split‐sample method. Data Sources and Study Setting Data were simulated to reflect a variety of real‐world quality measure distributions and scenarios. There is no date range to report as the data are simulated. Study Design Simulation studies of split‐sample reliability estimation were conducted under varying practical scenarios. Data Collection/Extraction Methods All data were simulated using functions in R. Principal Findings Single split‐sample reliability estimates can be very dependent on the random split of the data, especially in low sample size and low variability settings. Averaging split‐sample estimates over many splits of the data can yield a more stable reliability estimate. Conclusions Measure developers and evaluators using the split‐sample reliability method should average a series of reliability estimates calculated from many resamples of the data without replacement to obtain a more stable reliability estimate.