MVPGS: Excavating Multi-view Priors for Gaussian Splatting from Sparse Input Views
Recently, the Neural Radiance Field (NeRF) advancement has facilitated few-shot Novel View Synthesis (NVS), which is a significant challenge in 3D vision applications. Despite numerous attempts to reduce the dense input requirement in NeRF, it still suffers from time-consumed training and rendering...
Saved in:
Main Authors | , , , , , |
---|---|
Format | Journal Article |
Language | English |
Published |
22.09.2024
|
Subjects | |
Online Access | Get full text |
Cover
Loading…
Summary: | Recently, the Neural Radiance Field (NeRF) advancement has facilitated
few-shot Novel View Synthesis (NVS), which is a significant challenge in 3D
vision applications. Despite numerous attempts to reduce the dense input
requirement in NeRF, it still suffers from time-consumed training and rendering
processes. More recently, 3D Gaussian Splatting (3DGS) achieves real-time
high-quality rendering with an explicit point-based representation. However,
similar to NeRF, it tends to overfit the train views for lack of constraints.
In this paper, we propose \textbf{MVPGS}, a few-shot NVS method that excavates
the multi-view priors based on 3D Gaussian Splatting. We leverage the recent
learning-based Multi-view Stereo (MVS) to enhance the quality of geometric
initialization for 3DGS. To mitigate overfitting, we propose a forward-warping
method for additional appearance constraints conforming to scenes based on the
computed geometry. Furthermore, we introduce a view-consistent geometry
constraint for Gaussian parameters to facilitate proper optimization
convergence and utilize a monocular depth regularization as compensation.
Experiments show that the proposed method achieves state-of-the-art performance
with real-time rendering speed. Project page:
https://zezeaaa.github.io/projects/MVPGS/ |
---|---|
DOI: | 10.48550/arxiv.2409.14316 |