Deep Neural Networks Using Capsule Networks and Skeleton-Based Attentions for Action Recognition

• Published in: IEEE access Vol. 9; pp. 6164 - 6178
• Main Authors: Ha, Manh-Hung, Chen, Oscal Tzyh-Chiang
• Format: Journal Article
• Language: English
• Published: Piscataway IEEE 2021; The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
• Subjects: action recognition; Artificial neural networks; capsule network; Classifiers; convolutional neural network; Deep neural network; Feature extraction; Inference; Neural networks; Optical flow (image analysis); Optical imaging; Optical network units; Recognition; recurrent neural network; Recurrent neural networks; Skeleton; spatiotemporal attention; Three-dimensional displays; Video; Videos

This work develops Deep Neural Networks (DNNs) by adopting Capsule Networks (CapsNets) and spatiotemporal skeleton-based attention to effectively recognize subject actions from abundant spatial and temporal contexts of videos. The proposed generic DNN includes four 3D Convolutional Neural Networks (...