The distinction between discriminability and reliability in forensic science

• Published in: Science & justice Vol. 61; no. 4; pp. 319 - 331
• Main Authors: Smith, Andrew M., Neal, Tess M.S.
• Format: Journal Article
• Language: English
• Published: England Elsevier B.V 01.07.2021
• Subjects: Bayesian; Discriminability; Forecasting; Forensic Medicine; Forensic science; Forensic Sciences; Humans; Pathology; Reliability; Reproducibility of Results; ROC analysis; Signal detection theory; Signal detection theory; Discriminability; Forensic science; Reliability; ROC analysis; Bayesian
• ISSN: 1355-0306; 1876-4452; 1876-4452
• DOI: 10.1016/j.scijus.2021.04.002

•Scientists should primarily be concerned with maximizing discriminability.•The criminal justice system should primarily be concerned with estimating reliability.•ROC analysis informs on which of two procedures is superior, but percent correct does not.•Posterior by prior curves inform on how base rates moderate the reliability of evidence. Forensic science plays an increasingly important role in the criminal justice system; yet, many forensic procedures have not been subject to the empirical scrutiny that is expected in other scientific disciplines. Over the past two decades, the scientific community has done well to bridge the gap, but have likely only scratched the tip of the iceberg. We offer the discriminability-reliability distinction as a critical framework to guide future research on diagnostic-testing procedures in the forensic science domain. We argue that the primary concern of the scientist ought to be maximizing discriminability and that the primary concern of the criminal justice system ought to be assessing the reliability of evidence. We argue that Receiver Operating Characteristic (ROC) analysis is uniquely equipped for determining which of two procedures or conditions has better discriminability and we also demonstrate how estimates of reliability can be extracted from this Signal Detection framework.