The reproducibility crisis in the age of digital medicine

• Published in: NPJ digital medicine Vol. 2; no. 1; p. 2
• Main Authors: Stupple, Aaron, Singerman, David, Celi, Leo Anthony
• Format: Journal Article
• Language: English
• Published: London Nature Publishing Group UK 29.01.2019; Nature Publishing Group
• Subjects: 692/308/2778; 692/308/409; Algorithms; Artificial intelligence; Biomedicine; Biotechnology; Digital technology; Health informatics; Medical databases; Medical research; Medicine; Medicine & Public Health; Perspective; Reproducibility; Outcomes research; Preclinical research
• ISSN: 2398-6352; 2398-6352
• DOI: 10.1038/s41746-019-0079-z

As databases of medical information are growing, the cost of analyzing data is falling, and computer scientists, engineers, and investment are flooding into the field, digital medicine is subject to increasingly hyperbolic claims. Every week brings news of advances: superior algorithms that can predict clinical events and disease trajectory, classify images better than humans, translate clinical texts, and generate sensational discoveries around new risk factors and treatment effects. Yet the excitement about digital medicine—along with the technologies like the ones that enable a million people to watch a major event—poses risks for its robustness. How many of those new findings, in other words, are likely to be reproducible? Digital medicine must take steps to avoid a reproducibility “crisis” of the kind that has engulfed other areas of biomedicine and human science in the last decade and shaken public confidence in the validity of scientific work. The goal of this paper is to use a historical perspective on reproducibility and its current crisis to suggest how digital medicine can avoid a reproducibility crisis of its own.