Streaming Video QoE Modeling and Prediction: A Long Short-Term Memory Approach

• Published in: IEEE transactions on circuits and systems for video technology Vol. 30; no. 3; pp. 661 - 673
• Main Authors: Eswara, Nagabhushan, Ashique, S., Panchbhai, Anand, Chakraborty, Soumen, Sethuram, Hemanth P., Kuchi, Kiran, Kumar, Abhinav, Channappayya, Sumohana S.
• Format: Journal Article
• Language: English
• Published: New York IEEE 01.03.2020; The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
• Subjects: Adaptation models; Adaptive streaming; Datasets; Degradation; Hypertext; hypertext transfer protocol (HTTP); long short-term memory (LSTM); Modelling; Prediction models; Predictive models; Quality assessment; Quality of experience; quality-of-experience (QoE); rebuffering; Recurrent neural networks; recurrent neural networks (RNN); stalling; state space; State space models; Streaming media; time-varying quality; User satisfaction; video streaming
• ISSN: 1051-8215; 1558-2205
• DOI: 10.1109/TCSVT.2019.2895223

Due to the rate adaptation in hypertext transfer protocol adaptive streaming, the video quality delivered to the client keeps varying with time depending on the end-to-end network conditions. Moreover, the varying network conditions could also lead to the video client running out of the playback content resulting in rebuffering events. These factors affect the user satisfaction and cause degradation of the user quality of experience (QoE). Hence, it is important to quantify the perceptual QoE of the streaming video users and to monitor the same in a continuous manner so that the QoE degradation can be minimized. However, the continuous evaluation of QoE is challenging as it is determined by complex dynamic interactions among the QoE influencing factors. Toward this end, we present long short-term memory (LSTM)-QoE, a recurrent neural network-based QoE prediction model using an LSTM network. The LSTM-QoE is a network of cascaded LSTM blocks to capture the nonlinearities and the complex temporal dependencies involved in the time-varying QoE. Based on an evaluation over several publicly available continuous QoE datasets, we demonstrate that the LSTM-QoE has the capability to model the QoE dynamics effectively. We compare the proposed model with the state-of-the-art QoE prediction models and show that it provides an excellent performance across these datasets. Furthermore, we discuss the state space perspective for the LSTM-QoE and show the efficacy of the state space modeling approaches for the QoE prediction.