Free-Range Gaussians: Non-Grid-Aligned Generative 3D Gaussian Reconstruction

TL;DR

Free-Range Gaussians is a novel 3D reconstruction method that predicts non-grid-aligned Gaussians from few images, improving quality and reducing redundancy compared to prior grid-aligned approaches.

Contribution

The paper introduces a hierarchical patching scheme and a flow matching approach for non-grid-aligned 3D Gaussian reconstruction, enhancing fidelity and efficiency.

Findings

Outperforms pixel and voxel-aligned methods on Objaverse and Google Scanned Objects.

Uses fewer Gaussians while maintaining high-quality reconstructions.

Shows large improvements when input views leave parts of the object unobserved.

Abstract

We present Free-Range Gaussians, a multi-view reconstruction method that predicts non-pixel, non-voxel-aligned 3D Gaussians from as few as four images. This is done through flow matching over Gaussian parameters. Our generative formulation of reconstruction allows the model to be supervised with non-grid-aligned 3D data, and enables it to synthesize plausible content in unobserved regions. Thus, it improves on prior methods that produce highly redundant grid-aligned Gaussians, and suffer from holes or blurry conditional means in unobserved regions. To handle the number of Gaussians needed for high-quality results, we introduce a hierarchical patching scheme to group spatially related Gaussians into joint transformer tokens, halving the sequence length while preserving structure. We further propose a timestep-weighted rendering loss during training, and photometric gradient guidance and classifier-free guidance at inference to improve fidelity. Experiments on Objaverse and Google Scanned Objects show consistent improvements over pixel and voxel-aligned methods while using significantly fewer Gaussians, with large gains when input views leave parts of the object unobserved.