Flying with Photons: Rendering Novel Views of Propagating Light

We present an imaging and neural rendering technique that seeks to synthesize videos of light propagating through a scene from novel, moving camera viewpoints. Our approach relies on a new ultrafast imaging setup to capture a first-of-its kind, multi-viewpoint video dataset with picosecond-level tem...

Full description

Saved in:
Bibliographic Details
Published inarXiv.org
Main Authors Malik, Anagh, Juravsky, Noah, Ryan, Po, Wetzstein, Gordon, Kutulakos, Kiriakos N, Lindell, David B
Format Paper
LanguageEnglish
Published Ithaca Cornell University Library, arXiv.org 22.08.2024
Subjects
Cameras
Datasets
Light speed
Relativistic effects
Rendering
Specular reflection
Temporal resolution
Online AccessGet full text

Cover

More Information
Summary:We present an imaging and neural rendering technique that seeks to synthesize videos of light propagating through a scene from novel, moving camera viewpoints. Our approach relies on a new ultrafast imaging setup to capture a first-of-its kind, multi-viewpoint video dataset with picosecond-level temporal resolution. Combined with this dataset, we introduce an efficient neural volume rendering framework based on the transient field. This field is defined as a mapping from a 3D point and 2D direction to a high-dimensional, discrete-time signal that represents time-varying radiance at ultrafast timescales. Rendering with transient fields naturally accounts for effects due to the finite speed of light, including viewpoint-dependent appearance changes caused by light propagation delays to the camera. We render a range of complex effects, including scattering, specular reflection, refraction, and diffraction. Additionally, we demonstrate removing viewpoint-dependent propagation delays using a time warping procedure, rendering of relativistic effects, and video synthesis of direct and global components of light transport.
ISSN:2331-8422