A Phoneme-Informed Neural Network Model For Note-Level Singing Transcription

• Published in: ICASSP 2023 - 2023 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) pp. 1 - 5
• Main Authors: Yong, Sangeon, Su, Li, Nam, Juhan
• Format: Conference Proceeding
• Language: English
• Published: IEEE 04.06.2023
• Subjects: Feature extraction; Instruments; music information retrieval; Neural networks; Noise measurement; onset detection; phoneme classification; Phonetics; singing transcription; Task analysis; Timbre
• ISSN: 2379-190X
• DOI: 10.1109/ICASSP49357.2023.10096707

Note-level automatic music transcription is one of the most representative music information retrieval (MIR) tasks and has been studied for various instruments to understand music. However, due to the lack of high-quality labeled data, transcription of many instruments is still a challenging task. In particular, in the case of singing, it is difficult to find accurate notes due to its expressiveness in pitch, timbre, and dynamics. In this paper, we propose a method of finding note onsets of singing voice more accurately by leveraging the linguistic characteristics of singing, which are not seen in other instruments. The proposed model uses mel-scaled spectrogram and phonetic posteriorgram (PPG), a frame-wise likelihood of phoneme, as an input of the onset detection network while PPG is generated by the pre-trained network with singing and speech data. To verify how linguistic features affect onset detection, we compare the evaluation results through the dataset with different languages and divide onset types for detailed analysis. Our approach substantially improves the performance of singing transcription and therefore emphasizes the importance of linguistic features in singing analysis.