Real-Time Rendering of Point Clouds With Photorealistic Effects: A Survey

• Published in: IEEE access Vol. 10; pp. 13151 - 13173
• Main Authors: Kivi, Petrus E. J., Makitalo, Markku J., Zadnik, Jakub, Ikkala, Julius, Vadakital, Vinod Kumar Malamal, Jaaskelainen, Pekka O.
• Format: Journal Article
• Language: English
• Published: Piscataway IEEE 2022; The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
• Subjects: Acceleration; Hardware; photorealistic rendering; Point cloud compression; point clouds; point-based models; rasterization; Ray tracing; Real time; real-time rendering; Real-time systems; Rendering; Rendering (computer graphics); Solid modeling; Surface reconstruction; Survey; Three dimensional models
• ISSN: 2169-3536; 2169-3536
• DOI: 10.1109/ACCESS.2022.3146768

Readily available RGB-D cameras in smart phones and improving 3D scanning technologies have made it possible to produce detailed point cloud and point-based models of real world objects even in real time. Rendering such models in high quality and at satisfactory frame rates is needed for realistic extended reality (XR) applications. This publication reviews real-time photorealistic point cloud rendering methods which directly ray trace or rasterize point cloud models, with an emphasis on ray tracing and real-time performance. We found that real-time direct point cloud ray tracing research has been focused on static non-animated content, and thus, open research possibilities include adapting modern dedicated ray tracing hardware for increased performance for animated and live captured scenes, and adding path tracing techniques to increase photorealistic effects in the rendering result. A categorization and discussion on the capabilities of state-of-the-art photorealistic point cloud rendering methods is presented by surveying both real-time and offline methods, which are assumed to become real-time capable with the advances in near-future hardware. Challenges and future trends are derived by comparing different rasterization and ray tracing methods as well as acceleration structures for point clouds in terms of produced rendering effects and speed.