Recent developments in the Dorfman-Berbaum-Metz procedure for multireader ROC study analysis

• Published in: Academic radiology Vol. 15; no. 5; p. 647
• Main Authors: Hillis, Stephen L, Berbaum, Kevin S, Metz, Charles E
• Format: Journal Article
• Language: English
• Published: United States 01.05.2008
• Subjects: Adult; Algorithms; Analysis of Variance; Aneurysm, Dissecting - diagnosis; Aortic Aneurysm - diagnosis; Computer Simulation; Humans; Infant, Newborn; Infant, Newborn, Diseases - diagnosis; Radiography - statistics & numerical data; Radiology Information Systems; ROC Curve
• ISSN: 1076-6332
• DOI: 10.1016/j.acra.2007.12.015

The Dorfman-Berbaum-Metz (DBM) method has been one of the most popular methods for analyzing multireader receiver-operating characteristic (ROC) studies since it was proposed in 1992. Despite its popularity, the original procedure has several drawbacks: it is limited to jackknife accuracy estimates, it is substantially conservative, and it is not based on a satisfactory conceptual or theoretical model. Recently, solutions to these problems have been presented in three papers. Our purpose is to summarize and provide an overview of these recent developments. We present and discuss the recently proposed solutions for the various drawbacks of the original DBM method. We compare the solutions in a simulation study and find that they result in improved performance for the DBM procedure. We also compare the solutions using two real data studies and find that the modified DBM procedure that incorporates these solutions yields more significant results and clearer interpretations of the variance component parameters than the original DBM procedure. We recommend using the modified DBM procedure that incorporates the recent developments.