Skeleton Based Keyframe Detection Framework for Sports Action Analysis: Badminton Smash Case

• Published in: IEEE access Vol. 11; p. 1
• Main Authors: Sarwar, Muhammad Atif, Lin, Yu-Chen, Daraghmi, Yousef-Awwad, Ik, Tsi-Ui, Li, Yih-Lang
• Format: Journal Article
• Language: English
• Published: Piscataway IEEE 01.01.2023; The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
• Subjects: Action analysis; action analysis framework; Badminton; Cameras; Complexity; coordinate transform; Coordinate transformations; Data mining; Error detection; Feature extraction; Frames (data processing); Keyframe detection; Mean; Modules; Motion perception; Optimization; Pose estimation; Segments; Skeleton; skeleton detection; Sports; Three-dimensional displays; Video data; Videos
• ISSN: 2169-3536; 2169-3536
• DOI: 10.1109/ACCESS.2023.3307620

The analysis of badminton player actions from videos plays a crucial role in improving athletes' performance and generating statistical insights. The complexity and speed of badminton movements pose unique challenges compared to everyday activities. To analyze badminton player actions, we propose a skeleton-based keyframe detection framework for action analysis. Keyframe detection is widely used in video summarization and visual localization due to its computational efficiency and memory optimization compared to analyzing all frames of a video. This framework segments the complex macro-level activity into micro-level segments and analyzes each micro-level activity individually. Firstly, it extracts skeleton data from a motion sequence video using 3D:VIBE pose estimation. Then, the keyframe detection module explores the sequence of activity frames and identifies keyframes for each micro-level activity, including start, ready, strike, and end. Finally, the posture and movement detection modules analyze the posture and movement data to identify specific activities. This framework is implemented in the device called CoachBox. The proposed framework is evaluated using the mean absolute error on a dataset. The average mean absolute error for the keyframe detection module is less than 0.168 seconds, and the striking moment detection has an error of only 0.033 seconds. Additionally, a coordinate transform method is provided to convert body coordinates to real-world coordinates for visualization purposes.