Photorealistic rendering of mixed reality scenes

• Published in: Computer graphics forum Vol. 34; no. 2; pp. 643 - 665
• Main Authors: Kronander, Joel, Banterle, Francesco, Gardner, Andrew, Miandji, Ehsan, Unger, Jonas
• Format: Journal Article
• Language: English
• Published: Oxford Blackwell Publishing Ltd 01.05.2015
• Subjects: Analysis; Augmented reality; Categories and Subject Descriptors (according to ACM CCS); Computer graphics; I.3.3 [Computer Graphics]: Picture/Image Generation-Illumination Estimation; I.3.3 [Computer Graphics]: Picture/Image Generation—Illumination Estimation, Image‐Based Lighting, Reflectance and Shading; Illumination; Image processing systems; Image-Based Lighting; Lighting; Mixed reality; Photorealistic; Picture/Image Generation—Illumination Estimation; Reflectance and Shading; Rendering; Studies; Virtual reality; Vision systems
• ISSN: 0167-7055; 1467-8659; 1467-8659
• DOI: 10.1111/cgf.12591

Photo‐realistic rendering of virtual objects into real scenes is one of the most important research problems in computer graphics. Methods for capture and rendering of mixed reality scenes are driven by a large number of applications, ranging from augmented reality to visual effects and product visualization. Recent developments in computer graphics, computer vision, and imaging technology have enabled a wide range of new mixed reality techniques including methods for advanced image based lighting, capturing spatially varying lighting conditions, and algorithms for seamlessly rendering virtual objects directly into photographs without explicit measurements of the scene lighting. This report gives an overview of the state‐of‐the‐art in this field, and presents a categorization and comparison of current methods. Our in‐depth survey provides a tool for understanding the advantages and disadvantages of each method, and gives an overview of which technique is best suited to a specific problem.