A Semantic-Aware Transmission with Adaptive Control Scheme for Volumetric Video Service

• Published in: IEEE transactions on multimedia Vol. 25; pp. 1 - 13
• Main Authors: Zhu, Yuanwei, Huang, Yakun, Qiao, Xiuquan, Tan, Zhijie, Bai, Boyuan, Ma, Huadong, Dustdar, Schahram
• Format: Journal Article
• Language: English
• Published: Piscataway IEEE 01.01.2023; The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
• Subjects: Adaptation models; Adaptive control; adaptive transmission; Bandwidth; Bandwidths; Compression ratio; Feature extraction; Machine learning; Point cloud compression; Point cloud video; Real-time systems; reinforcement learning; Robust control; semantic-aware; Semantics; Streaming media; User experience; Video compression; Video data; Video transmission; Virtual reality
• ISSN: 1520-9210; 1941-0077
• DOI: 10.1109/TMM.2022.3217928

Volumetric video provides a more immersive holographic virtual experience than conventional video services such as 360-degree and virtual reality (VR) videos. However, due to ultra-high bandwidth requirements, existing compression and transmission technology cannot handle the delivery of real-time volumetric video. Unlike traditional compression methods and the approaches that extend 360-degree video streaming, we propose AITransfer, an AI-powered compression and semantic-aware transmission method for point cloud video data (a popular volumetric data format). AITransfer targets the semantic-level communication beyond transmitting raw point cloud video or compressed video with two outstanding contributions: (1) designing an integrated end-to-end architecture with two fundamental contents of feature extraction and reconstruction to reduce the bandwidth consumption and alleviate the computational pressure; and (2) incorporating the dynamic network condition into end-to-end architecture design and employing a deep reinforcement learning-based adaptive control scheme to provide robust transmission. We conduct extensive experiments on the typical datasets and develop a case study to demonstrate the efficiency and effectiveness. The results show that AITransfer can provide extremely efficient point cloud transmission while maintaining considerable user experience with more than 30.72x compression ratio under the existing network environments.