UniDrive: Towards Universal Driving Perception Across Camera Configurations

TL;DR

UniDrive introduces a unified virtual camera framework and optimization method to enable autonomous driving perception models to generalize across different camera configurations, improving adaptability and robustness.

Contribution

The paper proposes a novel virtual camera projection and configuration optimization approach that can be integrated into existing perception models for universal applicability.

Findings

Models trained on one camera configuration generalize well to others.

The virtual camera approach reduces performance degradation across configurations.

Experimental results on CARLA dataset validate the effectiveness of UniDrive.

Abstract

Vision-centric autonomous driving has demonstrated excellent performance with economical sensors. As the fundamental step, 3D perception aims to infer 3D information from 2D images based on 3D-2D projection. This makes driving perception models susceptible to sensor configuration (e.g., camera intrinsics and extrinsics) variations. However, generalizing across camera configurations is important for deploying autonomous driving models on different car models. In this paper, we present UniDrive, a novel framework for vision-centric autonomous driving to achieve universal perception across camera configurations. We deploy a set of unified virtual cameras and propose a ground-aware projection method to effectively transform the original images into these unified virtual views. We further propose a virtual configuration optimization method by minimizing the expected projection error between original and virtual cameras. The proposed virtual camera projection can be applied to existing 3D perception methods as a plug-and-play module to mitigate the challenges posed by camera parameter variability, resulting in more adaptable and reliable driving perception models. To evaluate the effectiveness of our framework, we collect a dataset on CARLA by driving the same routes while only modifying the camera configurations. Experimental results demonstrate that our method trained on one specific camera configuration can generalize to varying configurations with minor performance degradation.