Student Class Behavior Dataset: a video dataset for recognizing, detecting, and captioning students’ behaviors in classroom scenes

• Published in: Neural computing & applications Vol. 33; no. 14; pp. 8335 - 8354
• Main Authors: Sun, Bo, Wu, Yong, Zhao, Kaijie, He, Jun, Yu, Lejun, Yan, Huanqing, Luo, Ao
• Format: Journal Article
• Language: English
• Published: London Springer London 01.07.2021; Springer Nature B.V
• Subjects: Artificial Intelligence; Behavior; Classrooms; Computational Biology/Bioinformatics; Computational Science and Engineering; Computer Science; Data Mining and Knowledge Discovery; Datasets; Image Processing and Computer Vision; Modules; Original Article; Probability and Statistics in Computer Science; Psychology; Recognition; Students; Video data; Student Class Behavior; Video dataset; Baseline; Recognition dataset; Detection dataset; Caption dataset
• ISSN: 0941-0643; 1433-3058
• DOI: 10.1007/s00521-020-05587-y

The massive increase in classroom video data enables the possibility of utilizing artificial intelligence technology to automatically recognize, detect and caption students’ behaviors. This is beneficial for related research, e.g., pedagogy and educational psychology. However, the lack of a dataset specifically designed for students’ classroom behaviors may block these potential studies. This paper presents a comprehensive dataset that can be employed for recognizing, detecting, and captioning students’ behaviors in a classroom. We collected videos of 128 classes in different disciplines and in 11 classrooms. Specifically, the constructed dataset consists of a detection part, recognition part, and captioning part. The detection part includes a temporal detection data module with 4542 samples and an action detection data module with 3343 samples, whereas the recognition part contains 4276 samples and the captioning part contains 4296 samples. Moreover, the students’ behaviors are spontaneous in real classes, rendering the dataset representative and realistic. We analyze the special characteristics of the classroom scene and the technical difficulties for each module (task), which are verified by experiments. Due to the particularity of classrooms, our datasets proposes increasing the requirements of existing methods. Moreover, we provide a baseline for each task module in the dataset and make a comparison with the current mainstream datasets. The results show that our dataset is viable and reliable. Additionally, we present a thorough performance analysis of each baseline model to provide a comprehensive comparison for models using our presented dataset. The dataset and code are available to download online: https://github.com/BNU-Wu/Student-Class-Behavior-Dataset/tree/master .