Driver-Net: Multi-Camera Fusion for Assessing Driver Take-Over Readiness in Automated Vehicles

TL;DR

Driver-Net is a deep learning framework that fuses multi-camera visual cues to accurately and efficiently assess driver readiness for take-over in automated vehicles, improving safety and compliance.

Contribution

It introduces a novel multi-camera fusion architecture for driver monitoring, capturing synchronized cues from head, hands, and body to enhance prediction accuracy.

Findings

Achieves up to 95.8% accuracy in driver readiness classification.

Outperforms existing vision-based driver monitoring systems.

Demonstrates effectiveness on a diverse driving simulator dataset.

Abstract

Ensuring safe transition of control in automated vehicles requires an accurate and timely assessment of driver readiness. This paper introduces Driver-Net, a novel deep learning framework that fuses multi-camera inputs to estimate driver take-over readiness. Unlike conventional vision-based driver monitoring systems that focus on head pose or eye gaze, Driver-Net captures synchronised visual cues from the driver's head, hands, and body posture through a triple-camera setup. The model integrates spatio-temporal data using a dual-path architecture, comprising a Context Block and a Feature Block, followed by a cross-modal fusion strategy to enhance prediction accuracy. Evaluated on a diverse dataset collected from the University of Leeds Driving Simulator, the proposed method achieves an accuracy of up to 95.8% in driver readiness classification. This performance significantly enhances existing approaches and highlights the importance of multimodal and multi-view fusion. As a real-time, non-intrusive solution, Driver-Net contributes meaningfully to the development of safer and more reliable automated vehicles and aligns with new regulatory mandates and upcoming safety standards.