Learning to Score Figure Skating Sport Videos

• Published in: IEEE transactions on circuits and systems for video technology Vol. 30; no. 12; pp. 4578 - 4590
• Main Authors: Xu, Chengming, Fu, Yanwei, Zhang, Bing, Chen, Zitian, Jiang, Yu-Gang, Xue, Xiangyang
• Format: Journal Article
• Language: English
• Published: New York IEEE 01.12.2020; The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
• Subjects: Computational modeling; Convolutional codes; Datasets; Deep learning; Figure skating; Figure skating sport videos; Learning; multi-scale convolutional skip LSTM; self-attentive LSTM; Skating; Sports; Task analysis; Three-dimensional displays; Video; Videos
• ISSN: 1051-8215; 1558-2205
• DOI: 10.1109/TCSVT.2019.2927118

This paper aims at learning to score the figure skating sports videos. To address this task, we propose a deep architecture that includes two complementary components, i.e ., Self-Attentive LSTM and Multi-scale Convolutional Skip LSTM. These two components can efficiently learn the local and global sequential information in each video. Furthermore, we present a large-scale figure skating sports video dataset - FisV dataset. This dataset includes 500 figure skating videos with the average length of 2 minutes and 50 seconds. Each video is annotated by two scores of nine different referees, i.e., Total Element Score(TES) and Total Program Component Score (PCS). Our proposed model is validated on FisV and MIT-skate datasets. The experimental results show the effectiveness of our models in learning to score the figure skating videos. The codes and datasets would be downloaded from https://github.com/loadder/MS_LSTM.git .