Fuzzy Integral-Based CNN Classifier Fusion for 3D Skeleton Action Recognition

Action recognition based on skeleton key joints has gained popularity due to its cost effectiveness and low complexity. Existing Convolutional Neural Network (CNN) based models mostly fail to capture various aspects of the skeleton sequence. To this end, four feature representations, which capture c...

Full description

Saved in:
Bibliographic Details
Published inIEEE transactions on circuits and systems for video technology Vol. 31; no. 6; pp. 2206 - 2216
Main Authors Banerjee, Avinandan, Singh, Pawan Kumar, Sarkar, Ram
Format Journal Article
LanguageEnglish
Published New York IEEE 01.06.2021
The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
Subjects
3D skeleton
Action recognition
Artificial neural networks
Choquet integral
Classifiers
convolutional neural network
Cost effectiveness
Data mining
Feature extraction
fuzzy fusion
Image classification
Image coding
Integrals
Joints (anatomy)
Kinematics
Recognition
Skeleton
Source code
Three-dimensional displays
Two dimensional displays
Online AccessGet full text

Cover

More Information
Summary:Action recognition based on skeleton key joints has gained popularity due to its cost effectiveness and low complexity. Existing Convolutional Neural Network (CNN) based models mostly fail to capture various aspects of the skeleton sequence. To this end, four feature representations, which capture complementary characteristics of the sequence of key joints, are extracted with novel contribution of features estimated from angular information, and kinematics of the human actions. Single channel grayscale images are used to encode these features for classification using four CNNs, with the complementary nature verified through Kullback-Leibler (KL) and Jensen-Shannon (JS) divergences. As opposed to straightforward classifier combination generally used in existing literature, fuzzy fusion through the Choquet integral leverages the degree of uncertainty of decision scores obtained from four CNNs. Experimental results support the efficacy of fuzzy combination of CNNs to adaptively generate final decision score based upon confidence of each information source. Impressive results on the challenging UTD-MHAD, HDM05, G3D, and NTU RGB+D 60 and 120 datasets demonstrate the effectiveness of the proposed method. The source code for our method is available at https://github.com/theavicaster/fuzzy-integral-cnn-fusion-3d-har
Bibliography:ObjectType-Article-1
SourceType-Scholarly Journals-1
ObjectType-Feature-2
content type line 14
ISSN:1051-8215
1558-2205
DOI:10.1109/TCSVT.2020.3019293