Evaluating Whether a Binary Decision Rule Operates Better Than Chance

• Published in: Biometrical journal Vol. 41; no. 1; pp. 25 - 31
• Main Authors: Parker, Robert A., Davis, Roger B.
• Format: Journal Article
• Language: English
• Published: Berlin WILEY-VCH Verlag Berlin GmbH 01.03.1999; WILEY‐VCH Verlag Berlin GmbH; Wiley-VCH
• Subjects: Exact sciences and technology; Linear inference, regression; Mathematics; Nonparametric inference; Probability and statistics; Receiver-operating characteristic curve; ROC; Sciences and techniques of general use; Sensitivity; Specificity; Statistics; Youden's misclassification index; Receiver operating characteristic curves; Binary decision diagram; Confidence interval; Decision rule
• ISSN: 0323-3847; 1521-4036
• DOI: 10.1002/(SICI)1521-4036(199903)41:1<25::AID-BIMJ25>3.0.CO;2-A

We provide an approach to testing whether the accuracy of a binary diagnostic test, which we define as the sum of sensitivity and specificity, is significantly better than chance. We derive an exact confidence interval of size at least 1 — α for the observed accuracy of the test. In addition, we develop tests to compare the accuracy of two such tests applied to the same subjects. These results offer a method for assessing the accuracy of a test at a single test criterion, in contrast to the standard approach of evaluating the total receiver‐operating characteristic (ROC) curve for a test.