Drowsiness Detection Based On Driver Temporal Behavior Using a New Developed Dataset

TL;DR

This paper presents a real-time driver drowsiness detection system combining CNN and LSTM, utilizing a newly developed dataset that accounts for various disturbances, achieving high accuracy in identifying signs like yawning and blinking.

Contribution

The study introduces a novel dataset and a hybrid CNN-LSTM approach for improved real-time driver drowsiness detection under challenging conditions.

Findings

Effective detection of driver drowsiness using the hybrid CNN-LSTM model

Robustness of the system against illumination and head posture disturbances

Real-time performance demonstrated with multi-thread framework

Abstract

Driver drowsiness detection has been the subject of many researches in the past few decades and various methods have been developed to detect it. In this study, as an image-based approach with adequate accuracy, along with the expedite process, we applied YOLOv3 (You Look Only Once-version3) CNN (Convolutional Neural Network) for extracting facial features automatically. Then, LSTM (Long-Short Term Memory) neural network is employed to learn driver temporal behaviors including yawning and blinking time period as well as sequence classification. To train YOLOv3, we utilized our collected dataset alongside the transfer learning method. Moreover, the dataset for the LSTM training process is produced by the mentioned CNN and is formatted as a two-dimensional sequence comprised of eye blinking and yawning time durations. The developed dataset considers both disturbances such as illumination and drivers' head posture. To have real-time experiments a multi-thread framework is developed to run both CNN and LSTM in parallel. Finally, results indicate the hybrid of CNN and LSTM ability in drowsiness detection and the effectiveness of the proposed method.