3D skeleton‐based action recognition by representing motion capture sequences as 2D‐RGB images

• Published in: Computer animation and virtual worlds Vol. 28; no. 3-4
• Main Authors: Laraba, Sohaib, Brahimi, Mohammed, Tilmanne, Joëlle, Dutoit, Thierry
• Format: Journal Article
• Language: English
• Published: Chichester Wiley Subscription Services, Inc 01.05.2017
• Subjects: 3D data representation; action recognition; Algorithms; Artificial neural networks; Classification; Color imagery; convolutional neural networks; Human motion; Image classification; Machine learning; Motion capture; Moving object recognition; Neural networks; State of the art; Three dimensional motion; Trajectories
• DOI: 10.1002/cav.1782
• ISSN: 1546-4261

In recent years, 3D skeleton‐based action recognition has become a popular technique of action classification, thanks to development and availability of cheaper depth sensors. State‐of‐the‐art methods generally represent motion sequences as high dimensional trajectories followed by a time‐warping technique. These trajectories are used to train a classification model to predict the classes of new sequences. Despite the success of these techniques in some fields, particularly when the data used are captured by a high‐precision motion capture system, action classification is still less successful than the field of image classification, especially with the advance of deep learning. In this paper, we present a new representation of motion sequences (Seq2Im—for sequence to image), which projects motion sequences onto the RGB domain. The 3D coordinates of joints are mapped to red, green, and blue values, and therefore, action classification becomes an image classification problem and algorithms for this field can be applied. This representation was tested with basic image classification algorithms (namely, support vector machine, k‐nearest neighbor, and random forests) in addition to convolutional neural networks. Evaluation of the proposed method on standard 3D human action recognition datasets shows its potential for action recognition and outperforms most of the state‐of‐the‐art results. In this paper, we present a new representation of motion sequences (Seq2Im‐for sequence to image), which projects motion sequences onto the RGB domain. This representation was tested with basic image classification algorithms (namely, support vector machine, k‐nearest neighbor, and random forests) in addition to convolutional neural networks. Evaluation of the proposed method on standard 3D human action recognition datasets shows its potential for action recognition and outperforms most of the state‐of‐the‐art results.