A certified de-identification system for all clinical text documents for information extraction at scale

• Published in: JAMIA open Vol. 6; no. 3; p. ooad045
• Main Authors: Radhakrishnan, Lakshmi, Schenk, Gundolf, Muenzen, Kathleen, Oskotsky, Boris, Ashouri Choshali, Habibeh, Plunkett, Thomas, Israni, Sharat, Butte, Atul J
• Format: Journal Article
• Language: English
• Published: United States Oxford University Press 01.10.2023
• Subjects: Algorithms; Artificial intelligence; Certification; COVID-19; Data security; Electronic health records; Ethical aspects; Exercise; Health insurance; Health insurance industry; Health Insurance Portability & Accountability Act 1996-US; Identification systems; Information technology; Licensing, certification and accreditation; Medical history; Medical law; Medical records; Medical research; Medicine, Experimental; Pandemics; Patients' rights; Precision medicine; Privacy; Privacy, Right of; Research and Applications; Software; California; unstructured data; Philter; de-identification; clinical note text
• ISSN: 2574-2531; 2574-2531
• DOI: 10.1093/jamiaopen/ooad045

Abstract Objectives Clinical notes are a veritable treasure trove of information on a patient’s disease progression, medical history, and treatment plans, yet are locked in secured databases accessible for research only after extensive ethics review. Removing personally identifying and protected health information (PII/PHI) from the records can reduce the need for additional Institutional Review Boards (IRB) reviews. In this project, our goals were to: (1) develop a robust and scalable clinical text de-identification pipeline that is compliant with the Health Insurance Portability and Accountability Act (HIPAA) Privacy Rule for de-identification standards and (2) share routinely updated de-identified clinical notes with researchers. Materials and Methods Building on our open-source de-identification software called Philter, we added features to: (1) make the algorithm and the de-identified data HIPAA compliant, which also implies type 2 error-free redaction, as certified via external audit; (2) reduce over-redaction errors; and (3) normalize and shift date PHI. We also established a streamlined de-identification pipeline using MongoDB to automatically extract clinical notes and provide truly de-identified notes to researchers with periodic monthly refreshes at our institution. Results To the best of our knowledge, the Philter V1.0 pipeline is currently the first and only certified, de-identified redaction pipeline that makes clinical notes available to researchers for nonhuman subjects’ research, without further IRB approval needed. To date, we have made over 130 million certified de-identified clinical notes available to over 600 UCSF researchers. These notes were collected over the past 40 years, and represent data from 2757016 UCSF patients. Lay Summary Clinical notes and reports from routine patient care contain large amounts of clinically relevant information valuable for research like detailed diagnosis and treatment plans, over the counter medication usage, patient’s diet, and physical activity. Patient-level data are imperative to leverage for research, but access is restricted due to personal identifying (PII) and protected health information (PHI) content. Here we demonstrate how to anonymize the textual data by removing all PII/PHI following an externally certified protocol. The steps of the certification process and the detailed methodological enhancements are described. After many iterations of development and validation, the data were certified to be fully de-identified according to privacy laws. Secure access to this data can be granted internally in a safe and respectful way without compromising patient privacy. These efforts to provide valuable largely unexplored clinical notes data are a breakthrough for the biomedical research community. Having access to de-identified clinical data paves the way for artificial intelligence for medicine. We hope that our work would facilitate other institutions to incorporate our method and get computational understanding of clinical text.