GaussianCaR: Gaussian Splatting for Efficient Camera-Radar Fusion

TL;DR

GaussianCaR introduces an efficient camera-radar fusion method using Gaussian Splatting for BEV segmentation, achieving state-of-the-art performance with faster inference in autonomous vehicle perception.

Contribution

The paper presents GaussianCaR, a novel end-to-end network that leverages Gaussian Splatting for efficient camera-radar fusion in BEV segmentation tasks.

Findings

Achieves state-of-the-art IoU scores on nuScenes dataset.

Runs 3.2 times faster than previous methods.

Effectively combines multi-scale fusion with transformer decoding.

Abstract

Robust and accurate perception of dynamic objects and map elements is crucial for autonomous vehicles performing safe navigation in complex traffic scenarios. While vision-only methods have become the de facto standard due to their technical advances, they can benefit from effective and cost-efficient fusion with radar measurements. In this work, we advance fusion methods by repurposing Gaussian Splatting as an efficient universal view transformer that bridges the view disparity gap, mapping both image pixels and radar points into a common Bird's-Eye View (BEV) representation. Our main contribution is GaussianCaR, an end-to-end network for BEV segmentation that, unlike prior BEV fusion methods, leverages Gaussian Splatting to map raw sensor information into latent features for efficient camera-radar fusion. Our architecture combines multi-scale fusion with a transformer decoder to efficiently extract BEV features. Experimental results demonstrate that our approach achieves performance on par with, or even surpassing, the state of the art on BEV segmentation tasks (57.3%, 82.9%, and 50.1% IoU for vehicles, roads, and lane dividers) on the nuScenes dataset, while maintaining a 3.2x faster inference runtime. Code and project page are available online.