Optimal sampling for positive only electronic health record data

• Published in: Biometrics Vol. 79; no. 4; pp. 2974 - 2986
• Main Authors: Lee, Seong‐H., Ma, Yanyuan, Wei, Ying, Chen, Jinbo
• Format: Journal Article
• Language: English
• Published: United States Blackwell Publishing Ltd 01.12.2023
• Subjects: Classification; Electronic Health Records; Electronic medical records; False alarms; Humans; mean‐squared error; optimal sampling; Optimization; Phenotype; Phenotypes; Phenotyping; positive only; Sampling; Training; positive only; electronic health records; optimal sampling; mean-squared error
• ISSN: 0006-341X; 1541-0420
• DOI: 10.1111/biom.13824

Identifying a patient's disease/health status from electronic medical records is a frequently encountered task in electronic health records (EHR) related research, and estimation of a classification model often requires a benchmark training data with patients' known phenotype statuses. However, assessing a patient's phenotype is costly and labor intensive, hence a proper selection of EHR records as a training set is desired. We propose a procedure to tailor the best training subsample with limited sample size for a classification model, minimizing its mean‐squared phenotyping/classification error (MSE). Our approach incorporates “positive only” information, an approximation of the true disease status without false alarm, when it is available. In addition, our sampling procedure is applicable for training a chosen classification model which can be misspecified. We provide theoretical justification on its optimality in terms of MSE. The performance gain from our method is illustrated through simulation and a real‐data example, and is found often satisfactory under criteria beyond MSE.