Tuning model parameters in class‐imbalanced learning with precision‐recall curve

• Published in: Biometrical journal Vol. 61; no. 3; pp. 652 - 664
• Main Authors: Fu, Guang‐Hui, Yi, Lun‐Zhao, Pan, Jianxin
• Format: Journal Article
• Language: English
• Published: Germany Wiley - VCH Verlag GmbH & Co. KGaA 01.05.2019
• Subjects: Algorithms; Biometry - methods; Brain Injuries, Traumatic - diagnosis; Brain Injuries, Traumatic - metabolism; class imbalance; Colonic Neoplasms - diagnosis; Colonic Neoplasms - genetics; Colonic Neoplasms - physiopathology; Computer simulation; Humans; Learning; Machine Learning; Mathematical models; measurement; Models, Statistical; parameter tuning; Parameters; Performance measurement; precision‐recall curve; Recall; receiver operating characteristic; ROC Curve; Support Vector Machine; Tuning; class imbalance; parameter tuning; precision-recall curve; receiver operating characteristic; measurement
• ISSN: 0323-3847; 1521-4036; 1521-4036
• DOI: 10.1002/bimj.201800148

An issue for class‐imbalanced learning is what assessment metric should be employed. So far, precision‐recall curve (PRC) as a metric is rarely used in practice as compared with its alternative of receiver operating characteristic (ROC). This study investigates the performance of PRC as the evaluating criterion to address the class‐imbalanced data and focuses on the comparison of PRC with ROC. The advantages of PRC over ROC on assessing class‐imbalanced data are also investigated and tested on our proposed algorithm by tuning the whole model parameters in simulation studies and real data examples. The result shows that PRC is competitive with ROC as performance measurement for handling class‐imbalanced data in tuning the model parameters. PRC can be considered as an alternative but effective assessment for preprocessing (such as variable selection) skewed data and building a classifier in class‐imbalanced learning.