VGOS: Voxel Grid Optimization for View Synthesis from Sparse Inputs

TL;DR

VGOS is a fast method for reconstructing radiance fields from sparse views, using voxel grid optimization with regularization and incremental training to prevent overfitting and artifacts, achieving state-of-the-art results in minutes.

Contribution

The paper introduces VGOS, a novel voxel grid optimization approach with incremental training and regularization for efficient sparse-view view synthesis.

Findings

VGOS reconstructs radiance fields in 3-5 minutes from 3-10 views.

VGOS outperforms existing methods on sparse input scenarios.

VGOS achieves state-of-the-art quality with fast convergence.

Abstract

Neural Radiance Fields (NeRF) has shown great success in novel view synthesis due to its state-of-the-art quality and flexibility. However, NeRF requires dense input views (tens to hundreds) and a long training time (hours to days) for a single scene to generate high-fidelity images. Although using the voxel grids to represent the radiance field can significantly accelerate the optimization process, we observe that for sparse inputs, the voxel grids are more prone to overfitting to the training views and will have holes and floaters, which leads to artifacts. In this paper, we propose VGOS, an approach for fast (3-5 minutes) radiance field reconstruction from sparse inputs (3-10 views) to address these issues. To improve the performance of voxel-based radiance field in sparse input scenarios, we propose two methods: (a) We introduce an incremental voxel training strategy, which prevents overfitting by suppressing the optimization of peripheral voxels in the early stage of reconstruction. (b) We use several regularization techniques to smooth the voxels, which avoids degenerate solutions. Experiments demonstrate that VGOS achieves state-of-the-art performance for sparse inputs with super-fast convergence. Code will be available at https://github.com/SJoJoK/VGOS.