CATS-V2V: A Real-World Vehicle-to-Vehicle Cooperative Perception Dataset with Complex Adverse Traffic Scenarios

TL;DR

CATS-V2V is a comprehensive real-world dataset capturing complex adverse traffic scenarios for vehicle-to-vehicle cooperative perception, enabling improved autonomous driving AI through diverse, synchronized sensor data and annotations.

Contribution

The paper introduces CATS-V2V, the first large-scale dataset focusing on complex adverse traffic scenarios for V2V perception, with synchronized multi-modal data and a novel temporal alignment method.

Findings

Dataset covers 10 weather and lighting conditions across 10 locations.

Includes 60K LiDAR frames and 1.26M camera images with high-precision GNSS/IMU data.

Proposes a target-based temporal alignment method for sensor data synchronization.

Abstract

Vehicle-to-Vehicle (V2V) cooperative perception has great potential to enhance autonomous driving performance by overcoming perception limitations in complex adverse traffic scenarios (CATS). Meanwhile, data serves as the fundamental infrastructure for modern autonomous driving AI. However, due to stringent data collection requirements, existing datasets focus primarily on ordinary traffic scenarios, constraining the benefits of cooperative perception. To address this challenge, we introduce CATS-V2V, the first-of-its-kind real-world dataset for V2V cooperative perception under complex adverse traffic scenarios. The dataset was collected by two hardware time-synchronized vehicles, covering 10 weather and lighting conditions across 10 diverse locations. The 100-clip dataset includes 60K frames of 10 Hz LiDAR point clouds and 1.26M multi-view 30 Hz camera images, along with 750K anonymized yet high-precision RTK-fixed GNSS and IMU records. Correspondingly, we provide time-consistent 3D bounding box annotations for objects, as well as static scenes to construct a 4D BEV representation. On this basis, we propose a target-based temporal alignment method, ensuring that all objects are precisely aligned across all sensor modalities. We hope that CATS-V2V, the largest-scale, most supportive, and highest-quality dataset of its kind to date, will benefit the autonomous driving community in related tasks.