Rotation-Equivariant Conditional Spherical Neural Fields for Learning a Natural Illumination Prior
Inverse rendering is an ill-posed problem. Previous work has sought to resolve this by focussing on priors for object or scene shape or appearance. In this work, we instead focus on a prior for natural illuminations. Current methods rely on spherical harmonic lighting or other generic representation...
Saved in:
Main Authors | , , |
---|---|
Format | Journal Article |
Language | English |
Published |
07.06.2022
|
Subjects | |
Online Access | Get full text |
Cover
Loading…
Summary: | Inverse rendering is an ill-posed problem. Previous work has sought to
resolve this by focussing on priors for object or scene shape or appearance. In
this work, we instead focus on a prior for natural illuminations. Current
methods rely on spherical harmonic lighting or other generic representations
and, at best, a simplistic prior on the parameters. We propose a conditional
neural field representation based on a variational auto-decoder with a SIREN
network and, extending Vector Neurons, build equivariance directly into the
network. Using this, we develop a rotation-equivariant, high dynamic range
(HDR) neural illumination model that is compact and able to express complex,
high-frequency features of natural environment maps. Training our model on a
curated dataset of 1.6K HDR environment maps of natural scenes, we compare it
against traditional representations, demonstrate its applicability for an
inverse rendering task and show environment map completion from partial
observations. A PyTorch implementation, our dataset and trained models can be
found at jadgardner.github.io/RENI. |
---|---|
DOI: | 10.48550/arxiv.2206.03858 |