A numerical variability approach to results stability tests and its application to neuroimaging

Ensuring the long-term reproducibility of data analyses requires results stability tests to verify that analysis results remain within acceptable variation bounds despite inevitable software updates and hardware evolutions. This paper introduces a numerical variability approach for results stability...

Full description

Saved in:
Bibliographic Details
Published inIEEE transactions on computers pp. 1 - 10
Main Authors Chatelain, Yohan, Tetrel, Loic, Markiewicz, Christopher J., Goncalves, Mathias, Kiar, Gregory, Esteban, Oscar, Bellec, Pierre, Glatard, Tristan
Format Journal Article
LanguageEnglish
Published IEEE 07.10.2024
Subjects
Data analysis
Hardware
Magnetic resonance imaging
Neuroimaging
Numerical stability
Perturbation methods
Pipelines
Smoothing methods
Stability criteria
Online AccessGet full text

Cover

More Information
Summary:Ensuring the long-term reproducibility of data analyses requires results stability tests to verify that analysis results remain within acceptable variation bounds despite inevitable software updates and hardware evolutions. This paper introduces a numerical variability approach for results stability tests, which determines acceptable variation bounds using random rounding of floating-point calculations. By applying the resulting stability test to fMRIPrep , a widely-used neuroimaging tool, we show that the test is sensitive enough to detect subtle updates in image processing methods while remaining specific enough to accept numerical variations within a reference version of the application. This result contributes to enhancing the reliability and reproducibility of data analyses by providing a robust and flexible method for stability testing.
ISSN:0018-9340
1557-9956
DOI:10.1109/TC.2024.3475586