Training a Singing Transcription Model Using Connectionist Temporal Classification Loss and Cross-Entropy Loss

• Published in: IEEE/ACM transactions on audio, speech, and language processing Vol. 31; pp. 383 - 396
• Main Authors: Wang, Jun-You, Jang, Jyh-Shing Roger
• Format: Journal Article
• Language: English
• Published: Piscataway IEEE 2023; The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
• Subjects: Classification; Computational modeling; connectionist temporal classification loss; Data models; Datasets; Entropy; Hidden Markov models; Instruments; music information retrieval; Performance evaluation; Predictive models; Singing; Singing transcription; Task analysis; Training
• ISSN: 2329-9290; 2329-9304
• DOI: 10.1109/TASLP.2022.3224297

In this paper, we propose a method that uses a combination of the Connectionist Temporal Classification (CTC) loss and the cross-entropy loss to train a note-level singing transcription model. By considering the task as predicting a note sequence of the input audio, we can compute the CTC loss between the prediction and the groundtruth note sequence, and further use it with the traditional cross-entropy loss to optimize the transcription model. By comparing the proposed method with a baseline that only utilizes the cross-entropy loss, the results show improved model performance on all the evaluation metrics. Furthermore, using the CTC loss allows the transcription model to learn from weakly labeled data, which is easier to annotate than traditional strongly labeled data. Moreover, we point out the issue of the intrinsic global time shift on the onset labels between datasets. By automatically estimating and calibrating the global time shift of the training dataset, the performance of the singing transcription model is then not affected by the global time shift in the cross-dataset evaluation scenario.