Interpreting Incremental Value of Markers Added to Risk Prediction Models

The discrimination of a risk prediction model measures that model's ability to distinguish between subjects with and without events. The area under the receiver operating characteristic curve (AUC) is a popular measure of discrimination. However, the AUC has recently been criticized for its ins...

Full description

Saved in:
Bibliographic Details
Published inAmerican journal of epidemiology Vol. 176; no. 6; pp. 473 - 481
Main Authors Pencina, Michael J., D'Agostino, Ralph B., Pencina, Karol M., Janssens, A. Cecile J. W., Greenland, Philip
Format Journal Article
LanguageEnglish
Published Cary, NC Oxford University Press 15.09.2012
Oxford Publishing Limited (England)
Subjects
Adult
Aged
Aged, 80 and over
Atrial Fibrillation - diagnosis
Atrial Fibrillation - etiology
Atrial Fibrillation - metabolism
Biological and medical sciences
Biomarkers
Biomarkers - metabolism
Computer Simulation
Data Interpretation, Statistical
Epidemiology
General aspects
Humans
Logistic Models
Medical sciences
Middle Aged
Miscellaneous
Odds Ratio
Practice of Epidemiology
Prediction models
Prognosis
Public health. Hygiene
Public health. Hygiene-occupational medicine
Risk Assessment - methods
Risk Assessment - statistics & numerical data
Risk Factors
ROC Curve
risk assessment
risk factors
biomarkers
Prediction
Curve
Biological marker
Risk
Epidemiology
area under curve
Discrimination
Interpreting
Risk factor
Models
Predictive factor
Public health
Information interpretation
Online AccessGet full text

Cover

More Information
Summary:The discrimination of a risk prediction model measures that model's ability to distinguish between subjects with and without events. The area under the receiver operating characteristic curve (AUC) is a popular measure of discrimination. However, the AUC has recently been criticized for its insensitivity in model comparisons in which the baseline model has performed well. Thus, 2 other measures have been proposed to capture improvement in discrimination for nested models: the integrated discrimination improvement and the continuous net reclassification improvement. In the present study, the authors use mathematical relations and numerical simulations to quantify the improvement in discrimination offered by candidate markers of different strengths as measured by their effect sizes. They demonstrate that the increase in the AUC depends on the strength of the baseline model, which is true to a lesser degree for the integrated discrimination improvement. On the other hand, the continuous net reclassification improvement depends only on the effect size of the candidate variable and its correlation with other predictors. These measures are illustrated using the Framingham model for incident atrial fibrillation. The authors conclude that the increase in the AUC, integrated discrimination improvement, and net reclassification improvement offer complementary information and thus recommend reporting all 3 alongside measures characterizing the performance of the final model.
Bibliography:SourceType-Scholarly Journals-1
ObjectType-Feature-1
content type line 14
ObjectType-Article-1
ObjectType-Feature-2
content type line 23
Abbreviations: ΔAUC, change in area under the receiver operating characteristic curve; AUC, area under the receiver operating characteristic curve; BNP, B-type natriuretic peptide; CRP, C-reactive protein; IDI, integrated discrimination improvement; NRI, net reclassification improvement; NRI(>0), continuous net reclassification improvement.
ISSN:0002-9262
1476-6256
1476-6256
DOI:10.1093/aje/kws207