Consensus Synergizes with Memory: A Simple Approach for Anomaly Segmentation in Urban Scenes

Anomaly segmentation is a crucial task for safety-critical applications, such as autonomous driving in urban scenes, where the goal is to detect out-of-distribution (OOD) objects with categories which are unseen during training. The core challenge of this task is how to distinguish hard in-distribut...

Full description

Saved in:
Bibliographic Details
Published inarXiv.org
Main Authors Cen, Jiazhong, Jiang, Zenkun, Xie, Lingxi, Tian, Qi, Yang, Xiaokang, Shen, Wei
Format Paper
LanguageEnglish
Published Ithaca Cornell University Library, arXiv.org 24.11.2021
Subjects
Memory
Memory tasks
Object recognition
Psychology
Safety critical
Segmentation
Training
Online AccessGet full text

Cover

More Information
Summary:Anomaly segmentation is a crucial task for safety-critical applications, such as autonomous driving in urban scenes, where the goal is to detect out-of-distribution (OOD) objects with categories which are unseen during training. The core challenge of this task is how to distinguish hard in-distribution samples from OOD samples, which has not been explicitly discussed yet. In this paper, we propose a novel and simple approach named Consensus Synergizes with Memory (CosMe) to address this challenge, inspired by the psychology finding that groups perform better than individuals on memory tasks. The main idea is 1) building a memory bank which consists of seen prototypes extracted from multiple layers of the pre-trained segmentation model and 2) training an auxiliary model that mimics the behavior of the pre-trained model, and then measuring the consensus of their mid-level features as complementary cues that synergize with the memory bank. CosMe is good at distinguishing between hard in-distribution examples and OOD samples. Experimental results on several urban scene anomaly segmentation datasets show that CosMe outperforms previous approaches by large margins.
ISSN:2331-8422