Fisheye-GS: Lightweight and Extensible Gaussian Splatting Module for Fisheye Cameras

TL;DR

Fisheye-GS is a lightweight, extensible module that adapts Gaussian Splatting for fisheye cameras, improving visual quality and efficiency in 3D rendering with wide field of view lenses.

Contribution

We introduce a novel projection recalculation method for fisheye cameras that integrates seamlessly into existing 3D Gaussian Splatting frameworks, enhancing adaptability and performance.

Findings

Improved visual quality over undistortion-based methods

Seamless integration as a modular component

Enhanced efficiency for fisheye lens rendering

Abstract

Recently, 3D Gaussian Splatting (3DGS) has garnered attention for its high fidelity and real-time rendering. However, adapting 3DGS to different camera models, particularly fisheye lenses, poses challenges due to the unique 3D to 2D projection calculation. Additionally, there are inefficiencies in the tile-based splatting, especially for the extreme curvature and wide field of view of fisheye lenses, which are crucial for its broader real-life applications. To tackle these challenges, we introduce Fisheye-GS.This innovative method recalculates the projection transformation and its gradients for fisheye cameras. Our approach can be seamlessly integrated as a module into other efficient 3D rendering methods, emphasizing its extensibility, lightweight nature, and modular design. Since we only modified the projection component, it can also be easily adapted for use with different camera models. Compared to methods that train after undistortion, our approach demonstrates a clear improvement in visual quality.