Deep Analysis of CNN-based Spatio-temporal Representations for Action Recognition

In recent years, a number of approaches based on 2D or 3D convolutional neural networks (CNN) have emerged for video action recognition, achieving state-of-the-art results on several large-scale benchmark datasets. In this paper, we carry out in-depth comparative analysis to better understand the di...

Full description

Saved in:
Bibliographic Details
Published in2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) pp. 6161 - 6171
Main Authors Chen, Chun-Fu Richard, Panda, Rameswar, Ramakrishnan, Kandan, Feris, Rogerio, Cohn, John, Oliva, Aude, Fan, Quanfu
Format Conference Proceeding
LanguageEnglish
Published IEEE 01.06.2021
Subjects
Analytical models
Benchmark testing
Codes
Computer vision
Convolutional neural networks
Pattern recognition
Three-dimensional displays
Online AccessGet full text

Cover

More Information
Summary:In recent years, a number of approaches based on 2D or 3D convolutional neural networks (CNN) have emerged for video action recognition, achieving state-of-the-art results on several large-scale benchmark datasets. In this paper, we carry out in-depth comparative analysis to better understand the differences between these approaches and the progress made by them. To this end, we develop an unified framework for both 2D-CNN and 3D-CNN action models, which enables us to remove bells and whistles and provides a common ground for fair comparison. We then conduct an effort towards a large-scale analysis involving over 300 action recognition models. Our comprehensive analysis reveals that a) a significant leap is made in efficiency for action recognition, but not in accuracy; b) 2D-CNN and 3D-CNN models behave similarly in terms of spatio-temporal representation abilities and transferability. Our codes are available at https://github.com/IBM/action-recognition-pytorch.
ISSN:2575-7075
DOI:10.1109/CVPR46437.2021.00610