Combine and Conquer: A Meta-Analysis on Data Shift andOut-of-Distribution Detection

• Published in: Transactions on Machine Learning Research Journal
• Main Authors: Dadalto Câmara Gomes, Eduardo, Alberge, Florence, Duhamel, Pierre, Piantanida, Pablo
• Format: Journal Article
• Language: English
• Published: [Amherst Massachusetts]: OpenReview.net, 2022 01.07.2024
• Subjects: Artificial Intelligence; Computer Science
• ISSN: 2835-8856

This paper introduces a universal approach to seamlessly combine out-of-distribution (OOD)detection scores. These scores encompass a wide range of techniques that leverage the self-confidence of deep learning models and the anomalous behavior of features in the latentspace. Not surprisingly, combining such a varied population using simple statistics provesinadequate. To overcome this challenge, we propose a quantile normalization to map thesescores into p-values, effectively framing the problem into a multi-variate hypothesis test.Then, we combine these tests using established meta-analysis tools, resulting in a more effec-tive detector with consolidated decision boundaries. Furthermore, we create a probabilisticinterpretable criterion by mapping the final statistics into a distribution with known param-eters. Through empirical investigation, we explore different types of shifts, each exertingvarying degrees of impact on data. Our results demonstrate that our approach significantlyimproves overall robustness and performance across diverse OOD detection scenarios. No-tably, our framework is easily extensible for future developments in detection scores andstands as the first to combine decision boundaries in this context. The code and artifactsassociated with this work are publicly available.