Insights into the area under the receiver operating characteristic curve (AUC) as a discrimination measure in species distribution modelling

• Published in: Global ecology and biogeography Vol. 21; no. 4; pp. 498 - 507
• Main Author: Jiménez-Valverde, Alberto
• Format: Journal Article
• Language: English
• Published: Oxford, UK Blackwell Publishing Ltd 01.04.2012; Blackwell Publishing; Blackwell; Wiley Subscription Services, Inc
• Subjects: Animal and plant ecology; Animal, plant and microbial ecology; Applied ecology; AUC; background data; Biogeography; Biological and medical sciences; commission/omission errors; Conceptual lattices; Discrimination; Ecological modeling; environmental factors; Fall lines; Fundamental and applied biological sciences. Psychology; General aspects; General aspects. Techniques; MACROECOLOGICAL METHODS; Methods and techniques (sampling, tagging, trapping, modelling...); misclassification cost; Modeling; Population ecology; potential distribution; Predictive modeling; realized distribution; ROC curve; sensitivity; Simulations; Species; specificity; Synecology; threshold; Costs; Biogeography; Error; Ecology; threshold; realized distribution; Modeling; AUC; Specificity; Discrimination; Spatial distribution; Geographic distribution; commission/omission errors; background data; misclassification cost; ROC curve; sensitivity; potential distribution; Distribution range
• ISSN: 1466-822X; 1466-8238; 1466-822X
• DOI: 10.1111/j.1466-8238.2011.00683.x

Aim: The area under the receiver operating characteristic (ROC) curve (AUC) is a widely used statistic for assessing the discriminatory capacity of species distribution models. Here, I used simulated data to examine the interdependence of the AUC and classical discrimination measures (sensitivity and specificity) derived for the application of a threshold. I shall further exemplify with simulated data the implications of using the AUC to evaluate potential versus realized distribution models. Innovation: After applying the threshold that makes sensitivity and specificity equal, a strong relationship between the AUC and these two measures was found. This result is corroborated with real data. On the other hand, the AUC penalizes the models that estimate potential distributions (the regions where the species could survive and reproduce due to the existence of suitable environmental conditions), and favours those that estimate realized distributions (the regions where the species actually lives). Main conclusions: Firstly, the independence of the AUC from the threshold selection may be irrelevant in practice. This result also emphasizes the fact that the AUC assumes nothing about the relative costs of errors of omission and commission. However, in most real situations this premise may not be optimal. Measures derived from a contingency table for different cost ratio scenarios, together with the ROC curve, may be more informative than reporting just a single AUC value. Secondly, the AUC is only truly informative when there are true instances of absence available and the objective is the estimation of the realized distribution. When the potential distribution is the goal of the research, the AUC is not an appropriate performance measure because the weight of commission errors is much lower than that of omission errors.