MMDVS-LF: Multi-Modal Dynamic Vision Sensor and Eye-Tracking Dataset for Line Following

TL;DR

The paper introduces MMDVS-LF, a comprehensive multi-modal dataset combining DVS, eye-tracking, RGB, odometry, IMU, and demographic data to advance event-based deep learning for line following tasks.

Contribution

It presents the first dataset integrating multiple sensor modalities, specifically designed to facilitate development of models leveraging DVS data in control applications.

Findings

Enables new research in event-based deep learning algorithms.

Provides diverse sensor data for comprehensive analysis.

Facilitates development of models for control and perception tasks.

Abstract

Dynamic Vision Sensors (DVS) offer a unique advantage in control applications due to their high temporal resolution and asynchronous event-based data. Still, their adoption in machine learning algorithms remains limited. To address this gap and promote the development of models that leverage the specific characteristics of DVS data, we introduce the MMDVS-LF: Multi-Modal Dynamic Vision Sensor and Eye-Tracking Dataset for Line Following. This comprehensive dataset is the first to integrate multiple sensor modalities, including DVS recordings and eye-tracking data from a small-scale standardized vehicle. Additionally, the dataset includes RGB video, odometry, Inertial Measurement Unit (IMU) data, and demographic data of drivers performing a Line Following. With its diverse range of data, MMDVS-LF opens new opportunities for developing event-based deep learning algorithms just like the MNIST dataset did for Convolutional Neural Networks.