D-DASH: A Deep Q-Learning Framework for DASH Video Streaming

• Published in: IEEE transactions on cognitive communications and networking Vol. 3; no. 4; pp. 703 - 718
• Main Authors: Gadaleta, Matteo, Chiariotti, Federico, Rossi, Michele, Zanella, Andrea
• Format: Journal Article
• Language: English
• Published: IEEE 01.12.2017
• Subjects: Bit rate; DASH; deep Q-learning; Dynamic programming; Heuristic algorithms; HTTP adaptive streaming; LSTM; Neural networks; reinforcement learning; Streaming media
• ISSN: 2332-7731; 2332-7731
• DOI: 10.1109/TCCN.2017.2755007

The ever-increasing demand for seamless high-definition video streaming, along with the widespread adoption of the dynamic adaptive streaming over HTTP (DASH) standard, has been a major driver of the large amount of research on bitrate adaptation algorithms. The complexity and variability of the video content and of the mobile wireless channel make this an ideal application for learning approaches. Here, we present D-DASH, a framework that combines deep learning and reinforcement learning techniques to optimize the quality of experience (QoE) of DASH. Different learning architectures are proposed and assessed, combining feed-forward and recurrent deep neural networks with advanced strategies. D-DASH designs are thoroughly evaluated against prominent algorithms from the state-of-the-art, both heuristic and learning-based, evaluating performance indicators such as image quality across video segments and freezing/rebuffering events. Our numerical results are obtained on real and simulated channel traces and show the superiority of D-DASH in nearly all the considered quality metrics. Besides yielding a considerably higher QoE, the D-DASH framework exhibits faster convergence to the rate-selection strategy than the other learning algorithms considered in the study. This makes it possible to shorten the training phase, making D-DASH a good candidate for client-side runtime learning.