Transfer learning with fine tuning for human action recognition from still images

• Published in: Multimedia tools and applications Vol. 80; no. 13; pp. 20547 - 20578
• Main Authors: Chakraborty, Saikat, Mondal, Riktim, Singh, Pawan Kumar, Sarkar, Ram, Bhattacharjee, Debotosh
• Format: Journal Article
• Language: English
• Published: New York Springer US 01.05.2021; Springer Nature B.V
• Subjects: Artificial neural networks; Availability; Benchmarks; Computer Communication Networks; Computer Science; Computer vision; Data Structures and Information Theory; Datasets; Human activity recognition; Human motion; Machine learning; Multimedia Information Systems; Object recognition; Source code; Special Purpose and Application-Based Systems; Deep learning; Human action recognition; Stanford40; PPMI action dataset; Transfer learning; Still image
• ISSN: 1380-7501; 1573-7721
• DOI: 10.1007/s11042-021-10753-y

Still image-based human action recognition (HAR) is one of the most challenging research problems in the field of computer vision. Some of the significant reasons to support this claim are the availability of few datasets as well as fewer images per action class and the existence of many confusing classes in the datasets and comparing with video-based data. There is the unavailability of temporal information. In this work, we train some of the most reputed Convolutional Neural Network (CNN) based architectures using transfer learning after fine-tuned those suitably to develop a model for still image-based HAR. Since the number of images per action classes is found to be significantly less in number, we have also applied some well-known data augmentation techniques to increase the amount of data, which is always a need for deep learning-based models. Two benchmark datasets used for validating our model are Stanford 40 and PPMI, which are better known for their confusing action classes and the presence of occluded images and random poses of subjects. Results obtained by our model on these datasets outperform some of the benchmark results reported in the literature by a considerable margin. Class imbalance is deliberately introduced in the said datasets to better explore the robustness of the proposed model. The source code of the present work is available at: https://github.com/saikat021/Transfer-Learning-Based-HAR