Semi-Supervised Image-to-Video Adaptation for Video Action Recognition

• Published in: IEEE transactions on cybernetics Vol. 47; no. 4; pp. 960 - 973
• Main Authors: Zhang, Jianguang, Han, Yahong, Tang, Jinhui, Hu, Qinghua, Jiang, Jianmin
• Format: Journal Article
• Language: English
• Published: United States IEEE 01.04.2017; The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
• Subjects: Adaptation; Cameras; Computer science; Computer vision; Datasets; Feature extraction; human action recognition; Human activity recognition; Human motion; Image enhancement; Image recognition; Methods; Performance enhancement; Semantics; semi-supervised; State of the art; Training
• ISSN: 2168-2267; 2168-2275
• DOI: 10.1109/TCYB.2016.2535122

Human action recognition has been well explored in applications of computer vision. Many successful action recognition methods have shown that action knowledge can be effectively learned from motion videos or still images. For the same action, the appropriate action knowledge learned from different types of media, e.g., videos or images, may be related. However, less effort has been made to improve the performance of action recognition in videos by adapting the action knowledge conveyed from images to videos. Most of the existing video action recognition methods suffer from the problem of lacking sufficient labeled training videos. In such cases, over-fitting would be a potential problem and the performance of action recognition is restrained. In this paper, we propose an adaptation method to enhance action recognition in videos by adapting knowledge from images. The adapted knowledge is utilized to learn the correlated action semantics by exploring the common components of both labeled videos and images. Meanwhile, we extend the adaptation method to a semi-supervised framework which can leverage both labeled and unlabeled videos. Thus, the over-fitting can be alleviated and the performance of action recognition is improved. Experiments on public benchmark datasets and real-world datasets show that our method outperforms several other state-of-the-art action recognition methods.