DeepSense-V2V: A Vehicle-to-Vehicle Multi-Modal Sensing, Localization, and Communications Dataset

TL;DR

This paper introduces the first large-scale multi-modal dataset for vehicle-to-vehicle mmWave communication, capturing diverse scenarios with rich sensor data to facilitate research in V2V communication strategies.

Contribution

It provides a comprehensive, real-world dataset with multi-modal sensor data from a two-vehicle testbed, enabling advanced machine learning applications for V2V communication.

Findings

Dataset covers 120 km of driving in various conditions.

Contains over one million object detections.

Includes diverse sensor modalities like cameras, radars, lidar, and phased arrays.

Abstract

High data rate and low-latency vehicle-to-vehicle (V2V) communication are essential for future intelligent transport systems to enable coordination, enhance safety, and support distributed computing and intelligence requirements. Developing effective communication strategies, however, demands realistic test scenarios and datasets. This is important at the high-frequency bands where more spectrum is available, yet harvesting this bandwidth is challenged by the need for direction transmission and the sensitivity of signal propagation to blockages. This work presents the first large-scale multi-modal dataset for studying mmWave vehicle-to-vehicle communications. It presents a two-vehicle testbed that comprises data from a 360-degree camera, four radars, four 60 GHz phased arrays, a 3D lidar, and two precise GPSs. The dataset contains vehicles driving during the day and night for 120 km in intercity and rural settings, with speeds up to 100 km per hour. More than one million objects were detected across all images, from trucks to bicycles. This work further includes detailed dataset statistics that prove the coverage of various situations and highlights how this dataset can enable novel machine-learning applications.