Action Recognition and Localization by Hierarchical Space-Time Segments

• Published in: 2013 IEEE International Conference on Computer Vision pp. 2744 - 2751
• Main Authors: Shugao Ma, Jianming Zhang, Ikizler-Cinbis, Nazli, Sclaroff, Stan
• Format: Conference Proceeding Journal Article
• Language: English
• Published: IEEE 01.12.2013
• Subjects: action localization; action recognition; Color; Computer vision; Conferences; Hierarchies; Image segmentation; Localization; Motion segmentation; Position (location); Recognition; Representations; Roots; Segments; Shape; space-time representation; Tracking; Trajectory; Vegetation
• ISSN: 1550-5499
• DOI: 10.1109/ICCV.2013.341

We propose Hierarchical Space-Time Segments as a new representation for action recognition and localization. This representation has a two-level hierarchy. The first level comprises the root space-time segments that may contain a human body. The second level comprises multi-grained space-time segments that contain parts of the root. We present an unsupervised method to generate this representation from video, which extracts both static and non-static relevant space-time segments, and also preserves their hierarchical and temporal relationships. Using simple linear SVM on the resultant bag of hierarchical space-time segments representation, we attain better than, or comparable to, state-of-the-art action recognition performance on two challenging benchmark datasets and at the same time produce good action localization results.