Realistic Volume Rendering with Environment-Synced Illumination in Mixed Reality

Interactive volume visualization using a mixed reality (MR) system helps provide users with an intuitive spatial perception of volumetric data. Due to sophisticated requirements of user interaction and vision when using MR head-mounted display (HMD) devices, the conflict between the realisticness an...

Full description

Saved in:
Bibliographic Details
Published inarXiv.org
Main Authors Cheng, Haojie, Xu, Chunxiao, Chen, Xujing, Chen, Zhenxin, Wang, Jiajun, Zhao, Lingxiao
Format Paper
LanguageEnglish
Published Ithaca Cornell University Library, arXiv.org 05.09.2023
Subjects
Algorithms
Coherence
Digital video recorders
Display devices
Helmet mounted displays
Illumination
Mixed reality
Perception
Rendering
Visualization
Online AccessGet full text

Cover

More Information
Summary:Interactive volume visualization using a mixed reality (MR) system helps provide users with an intuitive spatial perception of volumetric data. Due to sophisticated requirements of user interaction and vision when using MR head-mounted display (HMD) devices, the conflict between the realisticness and efficiency of direct volume rendering (DVR) is yet to be resolved. In this paper, a new MR visualization framework that supports interactive realistic DVR is proposed. An efficient illumination estimation method is used to identify the high dynamic range (HDR) environment illumination captured using a panorama camera. To improve the visual quality of Monte Carlo-based DVR, a new spatio-temporal denoising algorithm is designed. Based on a reprojection strategy, it makes full use of temporal coherence between adjacent frames and spatial coherence between the two screens of an HMD to optimize MR rendering quality. Several MR development modules are also developed for related devices to efficiently and stably display the DVR results in an MR HMD. Experimental results demonstrate that our framework can better support immersive and intuitive user perception during MR viewing than existing MR solutions.
ISSN:2331-8422