GEN: Pushing the Limits of Softmax-Based Out-of-Distribution Detection

• Published in: 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) pp. 23946 - 23955
• Main Authors: Liu, Xixi, Lochman, Yaroslava, Zach, Christopher
• Format: Conference Proceeding
• Language: English
• Published: IEEE 01.06.2023
• Subjects: Deep architecture; Deep learning architectures and techniques; Feature extraction; Neural networks; Training; Training data; Transformers; Visualization
• ISSN: 2575-7075
• DOI: 10.1109/CVPR52729.2023.02293

Out-of-distribution (OOD) detection has been exten-sively studied in order to successfully deploy neural networks, in particular, for safety-critical applications. More-over, performing OOD detection on large-scale datasets is closer to reality, but is also more challenging. Sev-eral approaches need to either access the training data for score design or expose models to outliers during training. Some post-hoc methods are able to avoid the afore-mentioned constraints, but are less competitive. In this work, we propose Generalized ENtropy score (GEN), a simple but effective entropy-based score function, which can be applied to any pre-trained softmax-based classifier. Its performance is demonstrated on the large-scale ImageNet-lk OOD detection benchmark. It consistently improves the average AUROC across six commonly-used CNN-based and visual transformer classifiers over a num-ber of state-of-the-art post-hoc methods. The average AU- ROC improvement is at least 3.5%. Furthermore, we used GEN on top of feature-based enhancing methods as well as methods using training statistics to further improve the OOD detection performance. The code is available at: https://github.com/XixiLiu95/GEN.