CLACER: A Deep Learning-based Compilation Error Classification Method for Novice Students' Programs

• Published in: 2021 IEEE 45th Annual Computers, Software, and Applications Conference (COMPSAC) pp. 74 - 83
• Main Authors: Li, Zheng, Sun, Fuxiang, Wang, Haifeng, Ding, Yifan, Liu, Yong, Chen, Xiang
• Format: Conference Proceeding
• Language: English
• Published: IEEE 01.07.2021
• Subjects: Compilation error; Compilation error classification; Computational modeling; Deep learning; Location awareness; Neural networks; Novice students' programs; Predictive models; Program processors; Semantics; Static analysis
• DOI: 10.1109/COMPSAC51774.2021.00022

Compilation errors happen during the debugging process of novice students. Compiler error messages help novices to localize and remove errors, but these messages are difficult to understand for students. Some computing education researchers analyzed the compiler error messages generated by novice's attempts to compile their programs. However, some important questions remain open. For example, the existing compilation error category cannot cover all programs due to the simple static analysis and program repair patterns. And existing prediction models for classifying compilation errors are unsatisfactory because of the inappropriate neural networks. In this paper, we first propose a new category of compilation error based on the program tokens, which is the smallest unit of the program. Then we develop a neural network model CLACER (ClAssification of Compilation ERrors) based on TextCNN. CLACER performs better on extracting semantic features and statistical features from compiler error messages. To verify the effectiveness of our proposed category and corresponding method CLACER, we choose 16,926 student programs as our experimental subjects. Final experimental results indicate that our proposed classification category covers 16.5% more programs than the state-of-the-art category TEGCER. Moreover, CLACER improves the compiler's localization effectiveness and with a 4.25% improvement on the TEGCER category. Further analysis shows that CLACER has a promising prediction performance for different error classes, and TextCNN is more suitable for constructing the compilation error classification model.