A Model-Agnostic Algorithm for Bayes Error Determination in Binary Classification

This paper presents the intrinsic limit determination algorithm (ILD Algorithm), a novel technique to determine the best possible performance, measured in terms of the AUC (area under the ROC curve) and accuracy, that can be obtained from a specific dataset in a binary classification problem with ca...

Full description

Saved in:
Bibliographic Details
Published inAlgorithms Vol. 14; no. 11; p. 301
Main Authors Michelucci, Umberto, Sperti, Michela, Piga, Dario, Venturini, Francesca, Deriu, Marco A.
Format Journal Article
LanguageEnglish
Published Basel MDPI AG 01.11.2021
Subjects
Accuracy
Algorithms
area under the curve
binary classification
Classification
Datasets
intrinsic limits
Machine learning
Naïve Bayes classifier
Neural networks
ROC curve
Online AccessGet full text

Cover

More Information
Summary:This paper presents the intrinsic limit determination algorithm (ILD Algorithm), a novel technique to determine the best possible performance, measured in terms of the AUC (area under the ROC curve) and accuracy, that can be obtained from a specific dataset in a binary classification problem with categorical features regardless of the model used. This limit, namely, the Bayes error, is completely independent of any model used and describes an intrinsic property of the dataset. The ILD algorithm thus provides important information regarding the prediction limits of any binary classification algorithm when applied to the considered dataset. In this paper, the algorithm is described in detail, its entire mathematical framework is presented and the pseudocode is given to facilitate its implementation. Finally, an example with a real dataset is given.
ISSN:1999-4893
1999-4893
DOI:10.3390/a14110301