RIGI: Rectifying Image-to-3D Generation Inconsistency via Uncertainty-aware Learning

TL;DR

This paper introduces RIGI, a method that uses uncertainty-aware learning with 3D Gaussian Splatting to improve single-image 3D reconstruction by reducing artifacts caused by multi-view inconsistencies.

Contribution

We propose a novel uncertainty-aware regularization technique that dynamically detects and mitigates inconsistencies in multi-view 3D reconstruction from a single image.

Findings

Reduces artifacts and noise in 3D reconstructions

Improves consistency across multi-view images

Enhances overall 3D generation quality

Abstract

Given a single image of a target object, image-to-3D generation aims to reconstruct its texture and geometric shape. Recent methods often utilize intermediate media, such as multi-view images or videos, to bridge the gap between input image and the 3D target, thereby guiding the generation of both shape and texture. However, inconsistencies in the generated multi-view snapshots frequently introduce noise and artifacts along object boundaries, undermining the 3D reconstruction process. To address this challenge, we leverage 3D Gaussian Splatting (3DGS) for 3D reconstruction, and explicitly integrate uncertainty-aware learning into the reconstruction process. By capturing the stochasticity between two Gaussian models, we estimate an uncertainty map, which is subsequently used for uncertainty-aware regularization to rectify the impact of inconsistencies. Specifically, we optimize both Gaussian models simultaneously, calculating the uncertainty map by evaluating the discrepancies between rendered images from identical viewpoints. Based on the uncertainty map, we apply adaptive pixel-wise loss weighting to regularize the models, reducing reconstruction intensity in high-uncertainty regions. This approach dynamically detects and mitigates conflicts in multi-view labels, leading to smoother results and effectively reducing artifacts. Extensive experiments show the effectiveness of our method in improving 3D generation quality by reducing inconsistencies and artifacts.