FisheyeGaussianLift: BEV Feature Lifting for Surround-View Fisheye Camera Perception

TL;DR

This paper introduces FisheyeGaussianLift, a novel BEV segmentation method for fisheye cameras that models geometric uncertainty with Gaussian parameterization, enabling accurate perception without image undistortion.

Contribution

The proposed framework uniquely lifts fisheye image pixels into 3D space using Gaussian modeling and fuses them into BEV maps via differentiable splatting, bypassing the need for undistortion.

Findings

Achieves 87.75% IoU for drivable areas

Attains 57.26% IoU for vehicles

Performs robustly under severe fisheye distortion

Abstract

Accurate BEV semantic segmentation from fisheye imagery remains challenging due to extreme non-linear distortion, occlusion, and depth ambiguity inherent to wide-angle projections. We present a distortion-aware BEV segmentation framework that directly processes multi-camera high-resolution fisheye images,utilizing calibrated geometric unprojection and per-pixel depth distribution estimation. Each image pixel is lifted into 3D space via Gaussian parameterization, predicting spatial means and anisotropic covariances to explicitly model geometric uncertainty. The projected 3D Gaussians are fused into a BEV representation via differentiable splatting, producing continuous, uncertainty-aware semantic maps without requiring undistortion or perspective rectification. Extensive experiments demonstrate strong segmentation performance on complex parking and urban driving scenarios, achieving IoU scores of 87.75% for drivable regions and 57.26% for vehicles under severe fisheye distortion and diverse environmental conditions.