OmniGS: Fast Radiance Field Reconstruction using Omnidirectional Gaussian Splatting

TL;DR

OmniGS introduces a fast, GPU-accelerated omnidirectional Gaussian splatting system that reconstructs radiance fields from 360-degree images, enabling high-quality, real-time rendering without complex rectification.

Contribution

The paper presents a novel omnidirectional Gaussian splatting system with a theoretical analysis and GPU implementation, supporting direct splatting on equirectangular images for improved speed and quality.

Findings

Achieves state-of-the-art reconstruction quality.

Provides high rendering speed with omnidirectional images.

Supports differentiable optimization without cube-map rectification.

Abstract

Photorealistic reconstruction relying on 3D Gaussian Splatting has shown promising potential in various domains. However, the current 3D Gaussian Splatting system only supports radiance field reconstruction using undistorted perspective images. In this paper, we present OmniGS, a novel omnidirectional Gaussian splatting system, to take advantage of omnidirectional images for fast radiance field reconstruction. Specifically, we conduct a theoretical analysis of spherical camera model derivatives in 3D Gaussian Splatting. According to the derivatives, we then implement a new GPU-accelerated omnidirectional rasterizer that directly splats 3D Gaussians onto the equirectangular screen space for omnidirectional image rendering. We realize differentiable optimization of the omnidirectional radiance field without the requirement of cube-map rectification or tangent-plane approximation. Extensive experiments conducted in egocentric and roaming scenarios demonstrate that our method achieves state-of-the-art reconstruction quality and high rendering speed using omnidirectional images. The code will be publicly available.