M3CAD: Towards Generic Cooperative Autonomous Driving Benchmark

TL;DR

M3CAD is a comprehensive, multimodal benchmark for cooperative autonomous driving research, enabling evaluation of various tasks and fostering development of robust multi-vehicle systems.

Contribution

It introduces the most complete cooperative autonomous driving benchmark with multi-task, multimodal data and proposes an adaptive multi-level fusion method for efficient perception.

Findings

M3CAD supports diverse autonomous driving tasks with rich multimodal data.

Baseline evaluations demonstrate the benchmark's utility for assessing state-of-the-art methods.

The proposed fusion approach balances communication efficiency and perception accuracy.

Abstract

We introduce M$^3$CAD, a comprehensive benchmark designed to advance research in generic cooperative autonomous driving. M$^3$CAD comprises 204 sequences with 30,000 frames. Each sequence includes data from multiple vehicles and different types of sensors, e.g., LiDAR point clouds, RGB images, and GPS/IMU, supporting a variety of autonomous driving tasks, including object detection and tracking, mapping, motion forecasting, occupancy prediction, and path planning. This rich multimodal setup enables M$^3$CAD to support both single-vehicle and multi-vehicle cooperative autonomous driving research. To the best of our knowledge, M$^3$CAD is the most complete benchmark specifically designed for cooperative, multi-task autonomous driving research. To test its effectiveness, we use M$^3$CAD to evaluate both state-of-the-art single-vehicle and cooperative driving solutions, setting baseline performance results. Since most existing cooperative perception methods focus on merging features but often ignore network bandwidth requirements, we propose a new multi-level fusion approach which adaptively balances communication efficiency and perception accuracy based on the current network conditions. We release M$^3$CAD, along with the baseline models and evaluation results, to support the development of robust cooperative autonomous driving systems. All resources will be made publicly available on https://github.com/zhumorui/M3CAD