Bridging Diffusion Models and 3D Representations: A 3D Consistent Super-Resolution Framework

TL;DR

This paper introduces 3DSR, a super-resolution framework that uses diffusion models and 3D Gaussian-splatting to enhance 3D scene reconstructions with high visual quality and spatial consistency without extra fine-tuning.

Contribution

The paper presents a novel 3D super-resolution method leveraging diffusion models and Gaussian-splatting for explicit 3D consistency, outperforming prior implicit approaches.

Findings

Produces high-resolution, visually compelling 3D reconstructions

Maintains structural 3D consistency across views

Enhances visual quality without additional fine-tuning

Abstract

We propose 3D Super Resolution (3DSR), a novel 3D Gaussian-splatting-based super-resolution framework that leverages off-the-shelf diffusion-based 2D super-resolution models. 3DSR encourages 3D consistency across views via the use of an explicit 3D Gaussian-splatting-based scene representation. This makes the proposed 3DSR different from prior work, such as image upsampling or the use of video super-resolution, which either don't consider 3D consistency or aim to incorporate 3D consistency implicitly. Notably, our method enhances visual quality without additional fine-tuning, ensuring spatial coherence within the reconstructed scene. We evaluate 3DSR on MipNeRF360 and LLFF data, demonstrating that it produces high-resolution results that are visually compelling, while maintaining structural consistency in 3D reconstructions.