# Remote Tower Air Traffic Controller Multimodal Fatigue Detection

## TL;DR

A new system using eye and heart signals detects fatigue in remote air traffic controllers better than traditional methods.

## Contribution

A multimodal framework combining ocular and cardiac signals for fatigue detection in remote tower operations.

## Key findings

- The multimodal framework outperformed conventional models in fatigue detection.
- ECG/HRV and eye-tracking features complemented each other in capturing fatigue-related changes.
- Personalized calibration improved performance under cross-subject evaluation.

## Abstract

What are the main findings?
A multimodal framework integrating ocular and cardiac signals showed superior overall performance for fatigue detection in remote tower operations compared with conventional baseline models.ECG/HRV and eye-tracking features played complementary roles in characterizing fatigue-related physiological and behavioral changes.

A multimodal framework integrating ocular and cardiac signals showed superior overall performance for fatigue detection in remote tower operations compared with conventional baseline models.

ECG/HRV and eye-tracking features played complementary roles in characterizing fatigue-related physiological and behavioral changes.

What are the implications of the main findings?
The proposed strategy improved the detection of minority fatigue events under severe class imbalance.Personalized calibration helped mitigate cross-subject performance degradation and improved deployment readiness.

The proposed strategy improved the detection of minority fatigue events under severe class imbalance.

Personalized calibration helped mitigate cross-subject performance degradation and improved deployment readiness.

Remote tower (rTWR) operations are reshaping air traffic control but introduce significant human-factor risks, notably cognitive fatigue induced by prolonged screen-based visual surveillance. To mitigate these risks in a safety-critical domain where missed detections can be catastrophic, we propose a non-intrusive, multimodal fatigue detection framework fusing ocular and cardiac signals. A high-fidelity simulation study with 36 controllers was conducted to collect eye-tracking and electrocardiogram (ECG) data, from which a 12-dimensional feature vector—integrating gaze entropy and heart rate variability (HRV)—was extracted. Addressing the severe class imbalance and scarcity of fatigue samples in physiological data, we developed a cost-sensitive XGBoost classifier combining SMOTE oversampling with a dynamically weighted loss function. Experimental results show that the proposed framework performed well under mixed-subject evaluation and improved sensitivity to fatigue events. Although a marked performance drop was observed under LOSO evaluation, personalized calibration partially alleviated this limitation, indicating the potential of the framework for real-time fatigue monitoring in remote tower operations.