Skeleton Optical Spectra-Based Action Recognition Using Convolutional Neural Networks

• Published in: IEEE transactions on circuits and systems for video technology Vol. 28; no. 3; pp. 807 - 811
• Main Authors: Hou, Yonghong, Li, Zhaoyang, Wang, Pichao, Li, Wanqing
• Format: Journal Article
• Language: English
• Published: New York IEEE 01.03.2018; The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
• Subjects: Action recognition; Artificial neural networks; Body parts; Coding; Color texture; convolutional neural network (ConvNet); Encoding; Feature recognition; Image coding; Image color analysis; Image recognition; Joints (anatomy); Mapping; Neural networks; Skeleton; Training; Video data
• ISSN: 1051-8215; 1558-2205
• DOI: 10.1109/TCSVT.2016.2628339

This letter presents an effective method to encode the spatiotemporal information of a skeleton sequence into color texture images, referred to as skeleton optical spectra, and employs convolutional neural networks (ConvNets) to learn the discriminative features for action recognition. Such spectrum representation makes it possible to use a standard ConvNet architecture to learn suitable "dynamic" features from skeleton sequences without training millions of parameters afresh and it is especially valuable when there is insufficient annotated training video data. Specifically, the encoding consists of four steps: mapping of joint distribution, spectrum coding of joint trajectories, spectrum coding of body parts, and joint velocity weighted saturation and brightness. Experimental results on three widely used datasets have demonstrated the efficacy of the proposed method.