Self-Calibrating Gaussian Splatting for Large Field of View Reconstruction

TL;DR

This paper introduces a self-calibrating Gaussian Splatting framework that jointly optimizes camera parameters and scene representations, enabling high-quality large FOV scene reconstruction from fewer images with improved accuracy.

Contribution

It presents a novel hybrid network for modeling complex lens distortions and a cubemap resampling strategy, advancing scene reconstruction from wide-angle imagery.

Findings

Achieves state-of-the-art accuracy on synthetic datasets

Supports large FOV images without resolution loss

Compatible with fast Gaussian Splatting rasterization

Abstract

In this paper, we present a self-calibrating framework that jointly optimizes camera parameters, lens distortion and 3D Gaussian representations, enabling accurate and efficient scene reconstruction. In particular, our technique enables high-quality scene reconstruction from Large field-of-view (FOV) imagery taken with wide-angle lenses, allowing the scene to be modeled from a smaller number of images. Our approach introduces a novel method for modeling complex lens distortions using a hybrid network that combines invertible residual networks with explicit grids. This design effectively regularizes the optimization process, achieving greater accuracy than conventional camera models. Additionally, we propose a cubemap-based resampling strategy to support large FOV images without sacrificing resolution or introducing distortion artifacts. Our method is compatible with the fast rasterization of Gaussian Splatting, adaptable to a wide variety of camera lens distortion, and demonstrates state-of-the-art performance on both synthetic and real-world datasets.