On the application of ROC analysis to predict classification performance under varying class distributions

We counsel caution in the application of ROC analysis for prediction of classifier performance under varying class distributions. We argue that it is not reasonable to expect ROC analysis to provide accurate prediction of model performance under varying distributions if the classes contain causally...

Full description

Saved in:
Bibliographic Details
Published inMachine learning Vol. 58; no. 1; pp. 25 - 32
Main Authors WEBB, Geoffrey I, KAI MING TING
Format Journal Article
LanguageEnglish
Published Dordrecht Springer 2005
Springer Nature B.V
Subjects
Algorithms
Applied sciences
Artificial intelligence
Computer science
Computer science; control theory; systems
Exact sciences and technology
Learning and adaptive systems
Mathematics
Multivariate analysis
Probability and statistics
Sciences and techniques of general use
Statistics
Data distribution
Performance analysis
model evaluation
Correlation analysis
ROC analysis
Prediction
Online AccessGet full text

Cover

More Information
Summary:We counsel caution in the application of ROC analysis for prediction of classifier performance under varying class distributions. We argue that it is not reasonable to expect ROC analysis to provide accurate prediction of model performance under varying distributions if the classes contain causally relevant subclasses whose frequencies may vary at different rates or if there are attributes upon which the classes are causally dependent.[PUBLICATION ABSTRACT]
Bibliography:ObjectType-Article-2
SourceType-Scholarly Journals-1
ObjectType-Feature-1
content type line 23
ISSN:0885-6125
1573-0565
DOI:10.1007/s10994-005-4257-7