UniLION: Towards Unified Autonomous Driving Model with Linear Group RNNs

TL;DR

UniLION introduces a unified, efficient model for autonomous driving that handles various sensor data and tasks using linear group RNNs, achieving state-of-the-art results without complex fusion modules.

Contribution

The paper presents UniLION, a versatile architecture that unifies multiple autonomous driving tasks and sensor modalities with linear group RNNs, simplifying design and maintaining high performance.

Findings

Achieves state-of-the-art results in 3D perception tasks.

Supports multiple modalities and tasks within a single framework.

Reduces computational complexity compared to transformer-based models.

Abstract

Although transformers have demonstrated remarkable capabilities across various domains, their quadratic attention mechanisms introduce significant computational overhead when processing long-sequence data. In this paper, we present a unified autonomous driving model, UniLION, which efficiently handles large-scale LiDAR point clouds, high-resolution multi-view images, and even temporal sequences based on the linear group RNN operator (i.e., performs linear RNN for grouped features). Remarkably, UniLION serves as a single versatile architecture that can seamlessly support multiple specialized variants (i.e., LiDAR-only, temporal LiDAR, multi-modal, and multi-modal temporal fusion configurations) without requiring explicit temporal or multi-modal fusion modules. Moreover, UniLION consistently delivers competitive and even state-of-the-art performance across a wide range of core tasks, including 3D perception (e.g., 3D object detection, 3D object tracking, 3D occupancy prediction, BEV map segmentation), prediction (e.g., motion prediction), and planning (e.g., end-to-end planning). This unified paradigm naturally simplifies the design of multi-modal and multi-task autonomous driving systems while maintaining superior performance. Ultimately, we hope UniLION offers a fresh perspective on the development of 3D foundation models in autonomous driving. Code is available at https://github.com/happinesslz/UniLION