Research on automatic identification and evaluation method of piano playing skills based on convolutional neural network

• Published in: Applied mathematics and nonlinear sciences Vol. 10; no. 1
• Main Author: Wu, Xiaoliang
• Format: Journal Article
• Language: English
• Published: Beirut Sciendo 01.01.2025; De Gruyter Poland
• Subjects: 05C82; Dynamic temporal regularization (DTW) algorithm; G-HRNet model; High-resolution network HRNet; Piano playing; Playing techniques; YOLOv3 algorithm
• ISSN: 2444-8656; 2444-8656
• DOI: 10.2478/amns-2025-0622

This paper proposes a deep learning based piano hand fingering recognition system using YOLOv3 target detection algorithm for network training of the model. Process the images in the dataset and get the output of the network, use the trained model for target prediction. Based on this, a high-resolution network HRNet method is used to realize the recognition of piano playing techniques. At the same time, a Dynamic Time Warping (DTW) algorithm is introduced to calculate the similarity between playing techniques and standard techniques to complete the automatic evaluation. Finally, the performance dataset is utilized to verify the effectiveness of the recognition method in this paper. The results show that the G-HRNet model can effectively extract the angular features of the upper and lower joints of the player’s fingers, and its recognition accuracy is above 95% in both the training and test sets. In addition, the fluctuation of the pitch angle of the upper joints of the fingers at different times can be clearly seen using this model, and the recognition results are very clear. The scoring results for each finger of the subjects are realistic and can well reflect the flexibility and span of each finger. In the case evaluation results, the recognition accuracy of this paper’s model for video fingering and audio is above 85% and 90%, respectively. The evaluation standard of this fingering is reached.