Out-of-Distribution Detection by Cross-Class Vicinity Distribution of In-Distribution Data

• Published in: IEEE transaction on neural networks and learning systems Vol. 35; no. 10; pp. 1 - 12
• Main Authors: Zhao, Zhilin, Cao, Longbing, Lin, Kun-Yu
• Format: Journal Article
• Language: English
• Published: United States IEEE 01.10.2024
• Subjects: Deep learning; deep neural networks; Detectors; Image classification; mutual information; Neural networks; out-of-distribution detection; Training; Uncertainty; vicinity distribution
• ISSN: 2162-237X; 2162-2388; 2162-2388
• DOI: 10.1109/TNNLS.2023.3271856

Deep neural networks for image classification only learn to map in-distribution inputs to their corresponding ground-truth labels in training without differentiating out-of-distribution samples from in-distribution ones. This results from the assumption that all samples are independent and identically distributed (IID) without distributional distinction. Therefore, a pretrained network learned from in-distribution samples treats out-of-distribution samples as in-distribution and makes high-confidence predictions on them in the test phase. To address this issue, we draw out-of-distribution samples from the vicinity distribution of training in-distribution samples for learning to reject the prediction on out-of-distribution inputs. A cross-class vicinity distribution is introduced by assuming that an out-of-distribution sample generated by mixing multiple in-distribution samples does not share the same classes of its constituents. We, thus, improve the discriminability of a pretrained network by finetuning it with out-of-distribution samples drawn from the cross-class vicinity distribution, where each out-of-distribution input corresponds to a complementary label. Experiments on various in-/out-of-distribution datasets show that the proposed method significantly outperforms the existing methods in improving the capacity of discriminating between in-and out-of-distribution samples.