Augmenting bag-of-words: a robust contextual representation of spatiotemporal interest points for action recognition

• Published in: The Visual computer Vol. 31; no. 10; pp. 1383 - 1394
• Main Authors: Li, Yang, Ye, Junyong, Wang, Tongqing, Huang, Shijian
• Format: Journal Article
• Language: English
• Published: Berlin/Heidelberg Springer Berlin Heidelberg 01.10.2015; Springer Nature B.V
• Subjects: Artificial Intelligence; Coding; Computer Graphics; Computer Science; Human activity recognition; Image Processing and Computer Vision; Methods; Multilayers; Original Article; Representations; Spacetime; Statistical analysis; Action recognition; Cumulative probability histogram; Sparse coding; Contextual features
• ISSN: 0178-2789; 1432-2315
• DOI: 10.1007/s00371-014-1020-8

Although traditional bag-of-words model, together with local spatiotemporal features, has shown promising results for human action recognition, it ignores all structural information of features, which carries important information of motion structures in videos. Recent methods usually characterize the relationship of quantized spatiotemporal features to overcome this drawback. However, the propagation of quantization error leads to an unreliable representation. To alleviate the propagation of quantization error, we present a coding method, which considers not only the spatial similarity but also the reconstruction ability of visual words after giving a probabilistic interpretation of coding coefficients. Based on our coding method, a new type of feature called cumulative probability histogram is proposed to robustly characterize contextual structural information around interest points, which are extracted from multi-layered contexts and assumed to be complementary to local spatiotemporal features. The proposed method is verified on four benchmark datasets. Experiment results show that our method can achieve better performance than previous methods in action recognition.