meds_reader: A fast and efficient EHR processing library

The growing demand for machine learning in healthcare requires processing increasingly large electronic health record (EHR) datasets, but existing pipelines are not computationally efficient or scalable. In this paper, we introduce meds_reader, an optimized Python package for efficient EHR data proc...

Full description

Saved in:
Bibliographic Details
Main Authors Steinberg, Ethan, Wornow, Michael, Bedi, Suhana, Fries, Jason Alan, McDermott, Matthew B. A, Shah, Nigam H
Format Journal Article
LanguageEnglish
Published 12.09.2024
Subjects
Computer Science - Databases
Computer Science - Learning
Online AccessGet full text

Cover

More Information
Summary:The growing demand for machine learning in healthcare requires processing increasingly large electronic health record (EHR) datasets, but existing pipelines are not computationally efficient or scalable. In this paper, we introduce meds_reader, an optimized Python package for efficient EHR data processing that is designed to take advantage of many intrinsic properties of EHR data for improved speed. We then demonstrate the benefits of meds_reader by reimplementing key components of two major EHR processing pipelines, achieving 10-100x improvements in memory, speed, and disk usage. The code for meds_reader can be found at https://github.com/som-shahlab/meds_reader.
DOI:10.48550/arxiv.2409.09095