Hybrid‐360: An adaptive bitrate algorithm for tile‐based 360 video streaming

• Published in: Transactions on emerging telecommunications technologies Vol. 33; no. 4
• Main Authors: Yang, Shujie, Hu, Jialu, Jiang, Ke, Xiao, Han, Wang, Mu
• Format: Journal Article
• Language: English
• Published: 01.04.2022
• ISSN: 2161-3915; 2161-3915
• DOI: 10.1002/ett.4430

In recent years, virtual reality (VR) has become a popular topic. VR videos, as one of the popular applications, can provide the user with an extraordinary and immersed video viewing experience and are gradually entering the public's vision. However, it has faced specific difficulties in the past few years. VR videos need specific collection and displaying equipment. Delivering VR videos needs extra bandwidth and storage space as they contain more information. Traditional adaptive bitrate (ABR) algorithms are mainly designed for 2D videos, and cannot directly apply to VR videos. The delay must be neglectable and there should be fewer stalls. If not, users can suffer from motion sickness. With the development of relevant technologies, those problems can be solved. Using a first‐class VR camera and a head‐mounted display, VR videos can be easily collected and displayed properly. In this work, Hybrid‐360, a novel tile‐based 360 video ABR algorithm, is proposed. Using reinforcement learning, the algorithm can make proper bitrate decisions for every tile in the VR video based on the network and client status. In addition, a system model is proposed to implement the tile‐based 360 ABR algorithm in the real world. Different from traditional 2D videos, VR videos have special characteristics, which brings the need to redesign quality of experience (QoE) model. Additional QoE factors are added into the QoE model, and the weights of every term are adjusted. Using a novel 360 video simulation platform, Hybrid‐360 is then compared with some classical ABR algorithms. It is found to be capable of balancing the video quality as well as video smoothness and fluentness. Also, the algorithm can provide users with a higher QoE. In recent years, virtual reality (VR) has become more and more popular in many scenarios, such as gaming, health, sports, videos and so forth. VR video service can provide the user with an extraordinary and immersed video viewing experience, while it had faced particular difficulties. Omnidirectional video contains more data, which will cause high bandwidth consumption. As people can change their viewport arbitrarily, low delay is also required to prevent motion sickness. The stalls of video will significantly affect the quality of experience (QoE), so the adaptive bitrate (ABR) algorithm needs to make intelligent decisions. Fortunately, the development of 5G offers ultrahigh network throughput and ultralow delay. Also, with the help of reinforcement learning, the ABR algorithm can make better decisions to provide higher QoE. However, traditional ABR algorithms may encounter difficulties when the network condition becomes complex. In this work, a VR video delivery system is proposed to transmit tile‐based 360 videos. Considering user's viewport, the ABR module in the system can choose quality for different tiles. More specifically, a novel adaptive bitrate algorithm, Hybrid‐360, for tile‐based VR streaming is also proposed. The algorithm can find a balance between quality, smoothness, and fluentness. It uses asynchronous advantage actor‐critic algorithm and can take advantage of statistics in the past to make bitrate decisions. To reflect the user's watching experience more accurately, a novel QoE model is also established for VR videos. This algorithm is then compared with some well‐known algorithms. An improvement in QoE is found for Hybrid‐360.