DMD: A Large-Scale Multi-Modal Driver Monitoring Dataset for Attention and Alertness Analysis

TL;DR

This paper introduces DMD, a large-scale multi-modal driver monitoring dataset with diverse scenarios, and demonstrates its use in training real-time driver behavior recognition systems suitable for cost-efficient platforms.

Contribution

The paper presents the extensive DMD dataset and a novel real-time driver behavior recognition system optimized for CPU-only devices.

Findings

DMD dataset is more extensive and diverse than existing datasets.

The proposed system achieves high accuracy in real-time on CPU platforms.

Fusion strategies improve driver behavior recognition performance.

Abstract

Vision is the richest and most cost-effective technology for Driver Monitoring Systems (DMS), especially after the recent success of Deep Learning (DL) methods. The lack of sufficiently large and comprehensive datasets is currently a bottleneck for the progress of DMS development, crucial for the transition of automated driving from SAE Level-2 to SAE Level-3. In this paper, we introduce the Driver Monitoring Dataset (DMD), an extensive dataset which includes real and simulated driving scenarios: distraction, gaze allocation, drowsiness, hands-wheel interaction and context data, in 41 hours of RGB, depth and IR videos from 3 cameras capturing face, body and hands of 37 drivers. A comparison with existing similar datasets is included, which shows the DMD is more extensive, diverse, and multi-purpose. The usage of the DMD is illustrated by extracting a subset of it, the dBehaviourMD dataset, containing 13 distraction activities, prepared to be used in DL training processes. Furthermore, we propose a robust and real-time driver behaviour recognition system targeting a real-world application that can run on cost-efficient CPU-only platforms, based on the dBehaviourMD. Its performance is evaluated with different types of fusion strategies, which all reach enhanced accuracy still providing real-time response.