Surface-Aligned Neural Radiance Fields for Controllable 3D Human Synthesis

We propose a new method for reconstructing controllable implicit 3D human models from sparse multi-view RGB videos. Our method defines the neural scene representation on the mesh surface points and signed distances from the surface of a human body mesh. We identify an indistinguishability issue that...

Full description

Saved in:
Bibliographic Details
Published inarXiv.org
Main Authors Xu, Tianhan, Fujita, Yasuhiro, Matsumoto, Eiichi
Format Paper
LanguageEnglish
Published Ithaca Cornell University Library, arXiv.org 03.04.2022
Subjects
Finite element method
Interpolation
Representations
Synthesis
Three dimensional models
Online AccessGet full text

Cover

More Information
Summary:We propose a new method for reconstructing controllable implicit 3D human models from sparse multi-view RGB videos. Our method defines the neural scene representation on the mesh surface points and signed distances from the surface of a human body mesh. We identify an indistinguishability issue that arises when a point in 3D space is mapped to its nearest surface point on a mesh for learning surface-aligned neural scene representation. To address this issue, we propose projecting a point onto a mesh surface using a barycentric interpolation with modified vertex normals. Experiments with the ZJU-MoCap and Human3.6M datasets show that our approach achieves a higher quality in a novel-view and novel-pose synthesis than existing methods. We also demonstrate that our method easily supports the control of body shape and clothes. Project page: https://pfnet-research.github.io/surface-aligned-nerf/.
ISSN:2331-8422