On Data Classification Efficiency Based on a Trade-off Relation between Mutual Information and Error Probability

• Published in: 2020 International Conference on Information Technology and Nanotechnology (ITNT) pp. 1 - 6
• Main Authors: Lange, Mikhail, Lange, Andrey, Paramonov, Semion
• Format: Conference Proceeding
• Language: English
• Published: IEEE 26.05.2020
• Subjects: Data classification; decision algorithm; discriminant functions; error probability; error probability redundancy; lower bound; mutual information
• DOI: 10.1109/ITNT49337.2020.9253225

We propose a data classification model which yields an average mutual information between a set of objects and a set of class-label decisions as a function of error probability. Optimization of the model consists in minimization of the average mutual information by conditional distributions for the decisions subject to a given constraint on the average error probability. It is equivalent to calculating the rate-distortion function in a scheme of coding the source class labels with a given fidelity when a set of the class labels and a set of the objects are connected by an observation channel with known class-conditional probability distributions. Given set of the objects and known observation channel, a lower bound to the rate-distortion function is calculated. This bound is independent on a decision algorithm and yields a potentially achievable error probability subject to a fixed value of the average mutual information. The obtained bound is useful for evaluating an error probability redundancy of any decision algorithm with given discriminant functions.