Enhancement of 3D Point Cloud Contents Using 2D Image Super Resolution Network

• Published in: Journal of web engineering Vol. 21; no. 2; p. 425
• Main Authors: Park, Seonghwan, Kim, Junsik, Hwang, Yonghae, Suh, Doug Young, Kim, Kyuheon
• Format: Journal Article
• Language: English
• Published: Milan River Publishers 01.01.2022
• Subjects: Augmented reality; Codec; Data compression; Data loss; Deep learning; Dictionaries; Geometry; Image enhancement; Image reconstruction; Image sequencing; Neural networks; Spatial data; Three dimensional models; Video compression; Video data; Virtual reality
• ISSN: 1540-9589; 1544-5976
• DOI: 10.13052/jwe1540-9589.21213

Media technology has been developed to give users a sense of immersion. Recent media using 3D spatial data, such as augmented reality and virtual reality, has attracted attention. A point cloud is a data format that consists of a number of points, and thus can express 3D media using coordinates and color information for each point. Since a point cloud has a larger capacity than 2D images, a technology to compress the point cloud is required, i.e., standardized in the international standard organization MPEG as a video-based point cloud compression (V-PCC). V-PCC decomposes 3D point cloud data into 2D patches along orthogonal directions, and those patches are placed into a 2D image sequence, and then compressed using existing 2D video codecs. However, data loss may occur while converting a 3D point cloud into a 2D image sequence and encoding this sequence using a legacy video codec. This data loss can cause deterioration in the quality of a reconstructed point cloud. This paper proposed a method of enhancing a reconstructed point cloud by applying a super resolution network to the 2D patch image sequence of a 3D point cloud.