Learning and Matching of Dynamic Shape Manifolds for Human Action Recognition

• Published in: IEEE transactions on image processing Vol. 16; no. 6; pp. 1646 - 1661
• Main Authors: Liang Wang, Suter, D.
• Format: Journal Article
• Language: English
• Published: New York, NY IEEE 01.06.2007; Institute of Electrical and Electronics Engineers; The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
• Subjects: Action recognition; Algorithms; Applied sciences; Artificial Intelligence; Construction; Data mining; Detection, estimation, filtering, equalization, prediction; dimensionality reduction; Dynamics; Exact sciences and technology; Feature extraction; Human; human motion analysis; Humans; Image Enhancement - methods; Image Interpretation, Computer-Assisted - methods; Image processing; Imaging, Three-Dimensional - methods; Information, signal and communications theory; Kinematics; locality preserving projections (LPP); Locomotion - physiology; Manifolds; Miscellaneous; Motion analysis; Movement - physiology; Optical computing; Pattern recognition; Pattern Recognition, Automated - methods; Performance evaluation; Preserves; Recognition; Shape; Shape recognition; Signal and communications theory; Signal processing; Signal, noise; Similarity; Spatiotemporal phenomena; Studies; Task Performance and Analysis; Telecommunications and information theory; Tracking; Whole Body Imaging - methods; Nearest neighbour; Similarity; locality preserving projections (LPP); Pattern recognition; Partial occlusion; Learning; Image quality; Action recognition; dimensionality reduction; Signal processing; Motion analysis; human motion analysis; Pattern matching
• ISSN: 1057-7149; 1941-0042
• DOI: 10.1109/TIP.2007.896661

In this paper, we learn explicit representations for dynamic shape manifolds of moving humans for the task of action recognition. We exploit locality preserving projections (LPP) for dimensionality reduction, leading to a low-dimensional embedding of human movements. Given a sequence of moving silhouettes associated to an action video, by LPP, we project them into a low-dimensional space to characterize the spatiotemporal property of the action, as well as to preserve much of the geometric structure. To match the embedded action trajectories, the median Hausdorff distance or normalized spatiotemporal correlation is used for similarity measures. Action classification is then achieved in a nearest-neighbor framework. To evaluate the proposed method, extensive experiments have been carried out on a recent dataset including ten actions performed by nine different subjects. The experimental results show that the proposed method is able to not only recognize human actions effectively, but also considerably tolerate some challenging conditions, e.g., partial occlusion, low-quality videos, changes in viewpoints, scales, and clothes; within-class variations caused by different subjects with different physical build; styles of motion; etc