Real-Time Scene-Aware LiDAR Point Cloud Compression Using Semantic Prior Representation

• Published in: IEEE transactions on circuits and systems for video technology Vol. 32; no. 8; pp. 5623 - 5637
• Main Authors: Zhao, Lili, Ma, Kai-Kuang, Liu, Zhili, Yin, Qian, Chen, Jianwen
• Format: Journal Article
• Language: English
• Published: New York IEEE 01.08.2022; The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
• Subjects: Algorithms; Cloud computing; Coders; Encoding; Geometry; Image coding; Image segmentation; Labels; Laser radar; LiDAR; Localization; Object motion; Point cloud compression; Real time; Real-time systems; Representations; Semantics; Three dimensional models; Three-dimensional displays
• DOI: 10.1109/TCSVT.2022.3145513
• ISSN: 1051-8215

Existing LiDAR point cloud compression (PCC) methods tend to treat compression as a fidelity issue, without sufficiently addressing its machine perception aspect. The latter issue is often encountered by the decoder agents that might aim to conduct scene-understanding related tasks only, such as computing the localization information. For tackling this challenge, a novel LiDAR PCC system is proposed to compress the point cloud geometry, which contains a back channel for allowing the decoder to initiate such request to the encoder. The key success of our PCC method lies in our proposed semantic prior representation (SPR) and its lossy encoding algorithm with variable precision to generate the final bitstream; the entire process is fast and achieves real-time performance. Note that our SPR is a compact and effective representation of three-dimensional (3D) input point clouds, and it consists of labels, predictions , and residuals . These information can be generated by first exploiting a scene-aware object segmentation to a set of 2D range images (frames) individually, which were generated from the 3D point clouds via a projection process. Based on the generated labels, the pixels associated with those moving objects are considered as noisy information and should be removed for not only saving bit budget on transmission but also, most importantly, improving the accuracy of localization computed at the decoder. Experimental results conducted on the commonly-used test dataset have shown that our proposed system outperforms the MPEG's G-PCC (TMC13-v14.0) in a large bitrate range. In fact, the performance gap will become even larger when more and/or large moving objects are involved in the input point clouds.