Time-weighted motion history image for human activity classification in sports

• Published in: Sports engineering Vol. 26; no. 1
• Main Authors: Komori, Hideto, Isogawa, Mariko, Mikami, Dan, Nagai, Takasuke, Aoki, Yoshimitsu
• Format: Journal Article
• Language: English
• Published: London Springer London 01.12.2023; Springer Nature B.V
• Subjects: Biomedical Engineering and Bioengineering; Classification; Engineering; Engineering Design; Learning; Materials Science; Original Article; Rehabilitation Medicine; Representations; Sports; Sports Medicine; Theoretical and Applied Mechanics; Motion history image (MHI); Temporal template; Human activity classification
• ISSN: 1369-7072; 1460-2687
• DOI: 10.1007/s12283-023-00437-1

Vision-based human activity classification has remarkable potential for various applications in the sports context (e.g., motion analysis for performance enhancement, active sensing for athletes, etc.). Recently, learning-based human activity classifications have been widely researched. However, in sports scenes in which more detailed and player-specific classifications are required, this is a quite challenging task; in many cases, only a limited number of datasets are available, unlike daily movements such as walking or climbing stairs. Therefore, this paper proposes a time-weighted motion history image, an effective image sequence representation for learning-based human activity classification. Unlike conventional MHI based on the assumption that “the newer frame is more important,” our method generates importance-aware representation so that the predictor can “see” the frames that contribute to analyzing the specific human activity. Experimental results have shown the superiority of our method.