NeRF in the Wild: Neural Radiance Fields for Unconstrained Photo Collections

• Published in: Proceedings (IEEE Computer Society Conference on Computer Vision and Pattern Recognition. Online) pp. 7206 - 7215
• Main Authors: Martin-Brualla, Ricardo, Radwan, Noha, Sajjadi, Mehdi S. M., Barron, Jonathan T., Dosovitskiy, Alexey, Duckworth, Daniel
• Format: Conference Proceeding
• Language: English
• Published: IEEE 01.01.2021
• Subjects: Image color analysis; Learning systems; Lighting; Photorealism; Rendering (computer graphics); Solid modeling; Three-dimensional displays
• ISSN: 1063-6919
• DOI: 10.1109/CVPR46437.2021.00713

We present a learning-based method for synthesizing novel views of complex scenes using only unstructured collections of in-the-wild photographs. We build on Neural Radiance Fields (NeRF), which uses the weights of a multi-layer perceptron to model the density and color of a scene as a function of 3D coordinates. While NeRF works well on images of static subjects captured under controlled settings, it is incapable of modeling many ubiquitous, real-world phenomena in uncontrolled images, such as variable illumination or transient occluders. We introduce a series of extensions to NeRF to address these issues, thereby enabling accurate reconstructions from unstructured image collections taken from the internet. We apply our system, dubbed NeRF-W, to internet photo collections of famous landmarks, and demonstrate temporally consistent novel view renderings that are significantly closer to photorealism than the prior state of the art.