Development of a Bayesian Framework for Determining Uncertainty in Receiver Operating Characteristic Curve Estimates

• Published in: IEEE transactions on knowledge and data engineering Vol. 22; no. 1; pp. 31 - 45
• Main Authors: Parker, D.R., Gustafson, S.C., Oxley, M.E., Ross, T.D.
• Format: Journal Article
• Language: English
• Published: New York, NY IEEE 01.01.2010; IEEE Computer Society
• Subjects: Applied sciences; Bayesian methods; Biomedical imaging; Character generation; Computer science; control theory; systems; Data processing. List processing. Character string processing; Decision making; Exact sciences and technology; Focusing; Force sensors; Measurement; Memory organisation. Data processing; Object detection; Performance evaluation; performance metrics; receiver operating characteristic; Receivers; ROC curves; Software; target detection; Uncertainty; uncertainty estimation; Bayes estimation; Performance evaluation; Discriminant analysis; Modeling; ROC curves; uncertainty estimation; Probability density; Confidence; performance metrics; Receiver operating characteristic curves; Metric; receiver operating characteristic; Target detection
• DOI: 10.1109/TKDE.2009.50
• ISSN: 1041-4347

This research uses a Bayesian framework to develop probability densities for the receiver operating characteristic (ROC) curve. The ROC curve is a discrimination metric that may be used to quantify how well a detection system classifies targets and nontargets. The degree of uncertainty in ROC curve formulation is a concern that previous research has not adequately addressed. This research formulates a probability density for the ROC curve and characterizes its uncertainty using confidence bands. Methods for the generation and characterization of the probability densities of the ROC curve are specified and demonstrated, where the initial analysis employs beta densities to model target and nontarget samples of detection system output. For given target and nontarget data, given functional forms of the data densities (such as beta density forms) and given prior densities of the form parameters, the methods developed here provide exact performance metric probability densities.