CohortDiagnostics: Phenotype evaluation across a network of observational data sources using population-level characterization

• Published in: PloS one Vol. 20; no. 1; p. e0310634
• Main Authors: Rao, Gowtham A., Shoaibi, Azza, Makadia, Rupa, Hardin, Jill, Swerdel, Joel, Weaver, James, Voss, Erica A., Conover, Mitchell M., Fortin, Stephen, Sena, Anthony G., Knoll, Chris, Hughes, Nigel, Gilbert, James P., Blacketer, Clair, Andryc, Alan, DeFalco, Frank, Molinaro, Anthony, Reps, Jenna, Schuemie, Martijn J., Ryan, Patrick B.
• Format: Journal Article
• Language: English
• Published: United States Public Library of Science 16.01.2025; Public Library of Science (PLoS)
• Subjects: Algorithms; Alzheimer Disease - diagnosis; Alzheimer Disease - epidemiology; Alzheimer's disease; Biology and Life Sciences; Chronic conditions; Cohort Studies; Computer and Information Sciences; Data sources; Diabetes; Diagnosis; Female; Genotype & phenotype; Humans; Incidence; Information Sources; Laboratory tests; Lupus Erythematosus, Systemic - diagnosis; Lupus Erythematosus, Systemic - epidemiology; Male; Medical records; Medicine and Health Sciences; Middle Aged; Neurodegenerative diseases; Open source software; Phenotype; Phenotypes; Population; Population studies; Public domain; Public software; Source code; Systemic lupus erythematosus; Technology application; Trends; United States
• ISSN: 1932-6203; 1932-6203
• DOI: 10.1371/journal.pone.0310634

This paper introduces a novel framework for evaluating phenotype algorithms (PAs) using the open-source tool, Cohort Diagnostics. The method is based on several diagnostic criteria to evaluate a patient cohort returned by a PA. Diagnostics include estimates of incidence rate, index date entry code breakdown, and prevalence of all observed clinical events prior to, on, and after index date. We test our framework by evaluating one PA for systemic lupus erythematosus (SLE) and two PAs for Alzheimer's disease (AD) across 10 different observational data sources. By utilizing CohortDiagnostics, we found that the population-level characteristics of individuals in the cohort of SLE closely matched the disease's anticipated clinical profile. Specifically, the incidence rate of SLE was consistently higher in occurrence among females. Moreover, expected clinical events like laboratory tests, treatments, and repeated diagnoses were also observed. For AD, although one PA identified considerably fewer patients, absence of notable differences in clinical characteristics between the two cohorts suggested similar specificity. We provide a practical and data-driven approach to evaluate PAs, using two clinical diseases as examples, across a network of OMOP data sources. Cohort Diagnostics can ensure the subjects identified by a specific PA align with those intended for inclusion in a research study. Diagnostics based on large-scale population-level characterization can offer insights into the misclassification errors of PAs.