Driver distraction detection and recognition using RGB-D sensor

TL;DR

This paper presents a multi-modal RGB-D sensor system using Kinect to detect and recognize driver distraction types with high accuracy, integrating multiple behavioral modules and classification strategies for real-time in-car application.

Contribution

It introduces a novel multi-module system combining gaze, arm, head, and facial analysis with AdaBoost and HMM classifiers for driver distraction recognition.

Findings

85% accuracy in distraction type recognition

90% accuracy in distraction detection

Modules are independently usable for other inferences like fatigue detection

Abstract

Driver inattention assessment has become a very active field in intelligent transportation systems. Based on active sensor Kinect and computer vision tools, we have built an efficient module for detecting driver distraction and recognizing the type of distraction. Based on color and depth map data from the Kinect, our system is composed of four sub-modules. We call them eye behavior (detecting gaze and blinking), arm position (is the right arm up, down, right of forward), head orientation, and facial expressions. Each module produces relevant information for assessing driver inattention. They are merged together later on using two different classification strategies: AdaBoost classifier and Hidden Markov Model. Evaluation is done using a driving simulator and 8 drivers of different gender, age and nationality for a total of more than 8 hours of recording. Qualitative and quantitative results show strong and accurate detection and recognition capacity (85% accuracy for the type of distraction and 90% for distraction detection). Moreover, each module is obtained independently and could be used for other types of inference, such as fatigue detection, and could be implemented for real cars systems.