Graph-based Online Monitoring of Train Driver States via Facial and Skeletal Features

TL;DR

This paper introduces a novel online monitoring system for train driver alertness using a directed-graph neural network that combines facial and skeletal features, achieving high accuracy and including a new dataset with pathological conditions.

Contribution

The study develops a DGNN-based system that effectively classifies driver states and introduces a new dataset with simulated pathological conditions for railway safety.

Findings

Combining facial and skeletal features improves classification accuracy to 80.88%.

The system achieves over 99% accuracy in binary alertness detection.

A new dataset with pathological conditions enhances risk assessment capabilities.

Abstract

Driver fatigue poses a significant challenge to railway safety, with traditional systems like the dead-man switch offering limited and basic alertness checks. This study presents an online behavior-based monitoring system utilizing a customised Directed-Graph Neural Network (DGNN) to classify train driver's states into three categories: alert, not alert, and pathological. To optimize input representations for the model, an ablation study was performed, comparing three feature configurations: skeletal-only, facial-only, and a combination of both. Experimental results show that combining facial and skeletal features yields the highest accuracy (80.88%) in the three-class model, outperforming models using only facial or skeletal features. Furthermore, this combination achieves over 99% accuracy in the binary alertness classification. Additionally, we introduced a novel dataset that, for the first time, incorporates simulated pathological conditions into train driver monitoring, broadening the scope for assessing risks related to fatigue and health. This work represents a step forward in enhancing railway safety through advanced online monitoring using vision-based technologies.