Intelligent Caching for Mobile Video Streaming in Vehicular Networks with Deep Reinforcement Learning

• Published in: Applied sciences Vol. 12; no. 23; p. 11942
• Main Authors: Luo, Zhaohui, Liwang, Minghui
• Format: Journal Article
• Language: English
• Published: Basel MDPI AG 01.12.2022
• Subjects: Bandwidths; Caching; Data transmission; Deep learning; edge caching; High definition; Internet of Vehicles; Learning; Markov processes; mobile video; Mobility; multi-access edge computing; Optimization; Reinforcement; reinforcement learning; Reliability; Software; Storage capacity; Streaming; Streaming media; Streaming services; Vehicles; Video transmission
• ISSN: 2076-3417; 2076-3417
• DOI: 10.3390/app122311942

Caching-enabled multi-access edge computing (MEC) has attracted wide attention to support future intelligent vehicular networks, especially for delivering high-definition videos in the internet of vehicles with limited backhaul capacity. However, factors such as the constrained storage capacity of MEC servers and the mobility of vehicles pose challenges to caching reliability, particularly for supporting multiple bitrate video streaming caching while achieving considerable quality of experience (QoE). Motivated by the above challenges, in this paper, we propose an intelligent caching strategy that takes into account vehicle mobility, time-varying content popularity, and backhaul capability to improve the QoE of vehicle users effectively. First, based on the mobile video mean opinion score (MV-MOS), we designed an average download percentage (ADP) weighted QoE evaluation model. Then, the video content caching problem is formulated as a Markov decision process (MDP) to maximize the ADP weighted MV-MOS. Owing to the prior knowledge of video content popularity and channel state information that may not be available at the road side unit in practical scenarios, we propose a deep reinforcement learning (DRL)-based caching strategy to solve the problem while achieving a maximum ADP weighted MV-MOS. To accelerate its convergence speed, we further integrate the prioritized experience replay, dueling, and double deep Q-network technologies, which improve the performance of DRL algorithm. Numerical results demonstrate that the proposed DRL-based caching strategy significantly improves QoE, and achieves better video delivery reliability compared to existing non-learning approaches.