GaussianFusion: Gaussian-Based Multi-Sensor Fusion for End-to-End Autonomous Driving

TL;DR

GaussianFusion introduces a Gaussian-based multi-sensor fusion framework for end-to-end autonomous driving, enhancing interpretability and efficiency by using Gaussian representations to integrate multi-modal sensor data for improved trajectory planning.

Contribution

The paper proposes a novel Gaussian-based fusion method that employs explicit and implicit features for better multi-sensor integration in autonomous driving.

Findings

Outperforms existing fusion methods on NAVSIM and Bench2Drive benchmarks.

Provides more interpretable and robust sensor fusion results.

Enables iterative trajectory refinement through Gaussian interactions.

Abstract

Multi-sensor fusion is crucial for improving the performance and robustness of end-to-end autonomous driving systems. Existing methods predominantly adopt either attention-based flatten fusion or bird's eye view fusion through geometric transformations. However, these approaches often suffer from limited interpretability or dense computational overhead. In this paper, we introduce GaussianFusion, a Gaussian-based multi-sensor fusion framework for end-to-end autonomous driving. Our method employs intuitive and compact Gaussian representations as intermediate carriers to aggregate information from diverse sensors. Specifically, we initialize a set of 2D Gaussians uniformly across the driving scene, where each Gaussian is parameterized by physical attributes and equipped with explicit and implicit features. These Gaussians are progressively refined by integrating multi-modal features. The explicit features capture rich semantic and spatial information about the traffic scene, while the implicit features provide complementary cues beneficial for trajectory planning. To fully exploit rich spatial and semantic information in Gaussians, we design a cascade planning head that iteratively refines trajectory predictions through interactions with Gaussians. Extensive experiments on the NAVSIM and Bench2Drive benchmarks demonstrate the effectiveness and robustness of the proposed GaussianFusion framework. The source code will be released at https://github.com/Say2L/GaussianFusion.