Two-stream RNN/CNN for action recognition in 3D videos

The recognition of actions from video sequences has many applications in health monitoring, assisted living, surveillance, and smart homes. Despite advances in sensing, in particular related to 3D video, the methodologies to process the data are still subject to research. We demonstrate superior res...

Full description

Saved in:
Bibliographic Details
Published inProceedings of the ... IEEE/RSJ International Conference on Intelligent Robots and Systems pp. 4260 - 4267
Main Authors Rui Zhao, Ali, Haider, van der Smagt, Patrick
Format Conference Proceeding
LanguageEnglish
Published IEEE 01.09.2017
Subjects
Convolution
Logic gates
Recurrent neural networks
Skeleton
Three-dimensional displays
Training
Videos
Online AccessGet full text

Cover

More Information
Summary:The recognition of actions from video sequences has many applications in health monitoring, assisted living, surveillance, and smart homes. Despite advances in sensing, in particular related to 3D video, the methodologies to process the data are still subject to research. We demonstrate superior results by a system which combines recurrent neural networks with convolutional neural networks in a voting approach. The gated-recurrent-unit-based neural networks are particularly well-suited to distinguish actions based on long-term information from optical tracking data; the 3D-CNNs focus more on detailed, recent information from video data. The resulting features are merged in an SVM which then classifies the movement. In this architecture, our method improves recognition rates of state-of-the-art methods by 14% on standard data sets.
ISSN:2153-0866
DOI:10.1109/IROS.2017.8206288