Assessing the Effects of Delay to NMP via Audio Analysis

• Published in: SN computer science Vol. 4; no. 2; p. 126
• Main Authors: Tsioutas, Konstantinos, Xylomenos, George
• Format: Journal Article
• Language: English
• Published: Singapore Springer Nature Singapore 26.12.2022; Springer Nature B.V
• Subjects: Collaboration; Computer Imaging; Computer Science; Computer Systems Organization and Communication Networks; Data Structures and Information Theory; Delay; Feedback; Headphones; Information Systems and Communication Service; Musical instruments; Musical performances; Musicians & conductors; Original Research; Pattern Recognition and Graphics; Questionnaires; Signal Processing and Multimedia Applications; Software Engineering/Programming and Operating Systems; Synchronism; Vision; NMP; Audio delay; Tempo; QoE
• ISSN: 2661-8907; 2662-995X; 2661-8907
• DOI: 10.1007/s42979-022-01555-6

For network music performance (NMP), end-to-end delay is the most critical factor affecting the quality of experience (QoE) of the musicians, as longer delays prevent the musicians from synchronizing. To analyze the sensitivity of QoE to delay, we performed a controlled NMP experiment, where 11 pairs of musicians performed under a wide range of delays. The analysis of the QoE questionnaires answered by the participants produced results with wide variances, making the extraction of solid conclusions quite difficult. In this paper, we complement the subjective study with an analysis of the performance tempo of the NMP sessions. Specifically, we used signal processing techniques to analyze the audio recordings of the experiments, to recover the performance tempo of the musicians, assess its evolution during each session, and correlate it with the underlying delay. The results of the analysis indicate that musicians in real NMP settings are more tolerant to delay than previously thought, managing to reach and maintain a steady tempo even with one-way delays of 40 ms. We also study how the performance tempo is related to delay, finding that the exact relationship between the two depends on the musicians.