Enhanced Action Tubelet Detector for Spatio-Temporal Video Action Detection

Current spatio-temporal action detection methods usually employ a two-stream architecture, a RGB stream for raw images and an auxiliary motion stream for optical flow. Training is required individually for each stream and more efforts are necessary to improve the precision of RGB stream. To this end...

Full description

Saved in:
Bibliographic Details
Published inICASSP 2020 - 2020 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) pp. 2388 - 2392
Main Authors Wu, Yutang, Wang, Hanli, Wang, Shuheng, Li, Qinyu
Format Conference Proceeding
LanguageEnglish
Published IEEE 01.05.2020
Subjects
Action detection
conditional information
Detectors
Modulation
modulation layer
Optical flow
single stream network
Speech processing
Streaming media
Training
Visualization
Online AccessGet full text

Cover

More Information
Summary:Current spatio-temporal action detection methods usually employ a two-stream architecture, a RGB stream for raw images and an auxiliary motion stream for optical flow. Training is required individually for each stream and more efforts are necessary to improve the precision of RGB stream. To this end, a single stream network named enhanced action tubelet (EAT) detector is proposed in this work based on RGB stream. A modulation layer is designed to modulate RGB features with conditional information from the visual clues of optical flow and human pose. This network is end-to-end and the proposed layer can be easily applied into other action detectors. Experiments show that EAT detector outperforms traditional RGB stream and is competitive to existing two-stream methods while free from the trouble of training streams separately. By being embedded in a new three-stream architecture, the resulting three-stream EAT detector achieves impressive performances among the best competitors on UCF-Sports, JHMDB and UCF-101.
ISSN:2379-190X
DOI:10.1109/ICASSP40776.2020.9054394