Volume Sweeping: Learning Photoconsistency for Multi-View Shape Reconstruction

We propose a full study and methodology for multi-view stereo reconstruction with performance capture data. Multi-view 3D reconstruction has largely been studied with general, high resolution and high texture content inputs, where classic low-level feature extraction and matching are generally succe...

Full description

Saved in:
Bibliographic Details
Published inInternational journal of computer vision Vol. 129; no. 2; pp. 284 - 299
Main Authors Leroy, Vincent, Franco, Jean-Sébastien, Boyer, Edmond
Format Journal Article
LanguageEnglish
Published New York Springer US 01.02.2021
Springer
Springer Nature B.V
Springer Verlag
Subjects
Artificial Intelligence
Computer Imaging
Computer Science
Computer Vision and Pattern Recognition
Feature extraction
Image Processing and Computer Vision
Image reconstruction
Information management
Learning
Pattern Recognition
Pattern Recognition and Graphics
Sport clothes
Static objects
Sweeping
Texture
Visibility
Vision
Multi view stereo reconstruction
Volume sweeping
Learned photoconsistency
Performance capture
Learned Photoconsistency
Multi View Stereo Reconstruction
Performance Capture
Volume Sweeping
Online AccessGet full text

Cover

More Information
Summary:We propose a full study and methodology for multi-view stereo reconstruction with performance capture data. Multi-view 3D reconstruction has largely been studied with general, high resolution and high texture content inputs, where classic low-level feature extraction and matching are generally successful. However in performance capture scenarios, texture content is limited by wider angle shots resulting in smaller subject projection areas, and intrinsically low image content of casual clothing. We present a dedicated pipeline, based on a per-camera depth map sweeping strategy, analyzing in particular how recent deep network advances allow to replace classic multi-view photoconsistency functions with one that is learned. We show that learning based on a volumetric receptive field around a 3D depth candidate improves over using per-view 2D windows, giving the photoconsistency inference more visibility over local 3D correlations in viewpoint color aggregation. Despite being trained on a standard dataset of scanned static objects, the proposed method is shown to generalize and significantly outperform existing approaches on performance capture data, while achieving competitive results on recent benchmarks.
Bibliography:ObjectType-Article-1
SourceType-Scholarly Journals-1
ObjectType-Feature-2
content type line 14
ISSN:0920-5691
1573-1405
DOI:10.1007/s11263-020-01377-0