On Estimating Diagnostic Accuracy From Studies With Multiple Raters and Partial Gold Standard Evaluation

• Published in: Journal of the American Statistical Association Vol. 103; no. 481; pp. 61 - 73
• Main Authors: Albert, Paul S, Dodd, Lori E
• Format: Journal Article
• Language: English
• Published: Alexandria, VA Taylor & Francis 01.03.2008; American Statistical Association; Taylor & Francis Ltd
• Subjects: Accuracy; Applications; Applications and Case Studies; Calculus of variations and optimal control; Cancer; Diagnosis; Diagnostic error; Diagnostic errors; Digital technology; Diseases; Estimate reliability; Estimation bias; Exact sciences and technology; Gastrointestinal diseases; General topics; Gold standard; Illness; Latent class models; Mathematical analysis; Mathematics; Medical diagnosis; Medical errors; Medical imaging; Misclassification; Modeling; Parametric inference; Parametric models; Probability and statistics; Public health; Radiology; Sampling bias; Sciences and techniques of general use; Semilatent class models; Simulations; Standardized tests; Statistical analysis; Statistical data; Statistical methods; Statistical models; Statistics; Estimation error; Error estimation; Latent class models; Estimator robustness; Probability; Non parametric estimation; Statistical estimation; Probability distribution; Adaptive estimation; Diagnostic error; Statistical method; Diagnostic test; Simulation; Experimental design; Misclassification; Application; Latent class model; Biased estimation; Semilatent class models
• ISSN: 0162-1459; 1537-274X
• DOI: 10.1198/016214507000000329

We are often interested in estimating sensitivity and specificity of a group of raters or a set of new diagnostic tests in situations in which gold standard evaluation is expensive or invasive. Numerous authors have proposed latent modeling approaches for estimating diagnostic error without a gold standard. Albert and Dodd showed that, when modeling without a gold standard, estimates of diagnostic error can be biased when the dependence structure between tests is misspecified. In addition, they showed that choosing between different models for this dependence structure is difficult in most practical situations. While these results caution against using these latent class models, the difficulties of obtaining gold standard verification remain a practical reality. We extend two classes of models to provide a compromise that collects gold standard information on a subset of subjects but incorporates information from both the verified and nonverified subjects during estimation. We examine the robustness of diagnostic error estimation with this approach and show that choosing between competing models is easier in this context. In our analytic work and simulations, we consider situations in which verification is completely at random as well as settings in which the probability of verification depends on the actual test results. We apply our methodological work to a study designed to estimate the diagnostic error of digital radiography for gastric cancer.