Revisiting Open World Object Detection

• Published in: IEEE transactions on circuits and systems for video technology Vol. 34; no. 5; pp. 3496 - 3509
• Main Authors: Zhao, Xiaowei, Ma, Yuqing, Wang, Duorui, Shen, Yifan, Qiao, Yixuan, Liu, Xianglong
• Format: Journal Article
• Language: English
• Published: New York IEEE 01.05.2024; The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
• Subjects: Algorithms; auxiliary proposal advisor; Benchmark testing; Benchmarks; class-specific expelling classifier; Labeling; Measurement; Object detection; Object recognition; Open world; Principles; Proposals; Task analysis; Training
• ISSN: 1051-8215; 1558-2205
• DOI: 10.1109/TCSVT.2023.3326279

Open World Object Detection (OWOD), simulating the real dynamic world where knowledge grows continuously, attempts to detect both known and unknown classes and incrementally learn the identified unknown ones. Recently a few studies have introduced and explored the OWOD problem, however, the main challenges in the OWOD task that distinguishing unknown classes from the background (Unknown Objectness) or known classes (Unknown Discrimination) have not been well solved, and there is lacking systematic analysis of benchmark and metrics for evaluating the OWOD task. In this paper, we revisit the OWOD problem and rethink it from benchmark, metrics, and algorithm perspectives. First, we propose five fundamental benchmark principles in line with the OWOD definition and construct two OWOD benchmarks according to the principles for a fair evaluation. Second, we point out that existing metrics neglect the detection performance of unknown classes and further design two additional metrics specific to the OWOD problem, filling the void of evaluating from the perspective of unknown classes. Finally, we introduce a novel and effective OWOD framework with an auxiliary Proposal ADvisor (PAD) and a Class-specific Expelling Classifier (CEC). The non-parametric PAD improves Unknown Objectness by assisting RPN in identifying more accurate unknown proposals based on the class-agnostic property of the object and aggregation through spatial and appearance similarity, while CEC enhances the Unknown Discrimination by calibrating the over-confident activation boundary and suppressing confusing predictions through a class-specific expelling function. Comprehensive experiments conducted on both fair benchmarks based on our OWOD benchmark principles and the original benchmark demonstrate that our method outperforms other state-of-the-art object detection methods in terms of both existing and our new metrics.