T-CNN: Tubelets With Convolutional Neural Networks for Object Detection From Videos

• Published in: IEEE transactions on circuits and systems for video technology Vol. 28; no. 10; pp. 2896 - 2907
• Main Authors: Kang, Kai, Li, Hongsheng, Yan, Junjie, Zeng, Xingyu, Yang, Bin, Xiao, Tong, Zhang, Cong, Wang, Zhe, Wang, Ruohui, Wang, Xiaogang, Ouyang, Wanli
• Format: Journal Article
• Language: English
• Published: New York IEEE 01.10.2018; The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
• Subjects: Artificial neural networks; computer vision; Convolutional codes; Image detection; Machine learning; Neural networks; Object detection; Object recognition; Proposals; State of the art; Target tracking; Training; Trucks; Videos
• ISSN: 1051-8215; 1558-2205
• DOI: 10.1109/TCSVT.2017.2736553

The state-of-the-art performance for object detection has been significantly improved over the past two years. Besides the introduction of powerful deep neural networks, such as GoogleNet and VGG, novel object detection frameworks, such as R-CNN and its successors, Fast R-CNN, and Faster R-CNN, play an essential role in improving the state of the art. Despite their effectiveness on still images, those frameworks are not specifically designed for object detection from videos. Temporal and contextual information of videos are not fully investigated and utilized. In this paper, we propose a deep learning framework that incorporates temporal and contextual information from tubelets obtained in videos, which dramatically improves the baseline performance of existing still-image detection frameworks when they are applied to videos. It is called T-CNN, i.e., tubelets with convolutional neueral networks. The proposed framework won newly introduced an object-detection-from-video task with provided data in the ImageNet Large-Scale Visual Recognition Challenge 2015. Code is publicly available at https://github.com/myfavouritekk/T-CNN .