A Neuro-Symbolic System for Interpretable Multimodal Physiological Signals Integration in Human Fatigue Detection

TL;DR

This paper introduces a neuro-symbolic system that combines interpretable physiological concepts and differentiable reasoning for fatigue detection from multimodal signals, achieving competitive accuracy and transparency.

Contribution

The work presents a novel neuro-symbolic architecture that learns interpretable physiological concepts and integrates them with reasoning rules for fatigue classification.

Findings

Achieves 72.1% accuracy on leave-one-subject-out evaluation

Exposes concept activations and rule firing strengths for interpretability

Introduces concept fidelity metric strongly correlated with accuracy

Abstract

We propose a neuro-symbolic architecture that learns four interpretable physiological concepts, oculomotor dynamics, gaze stability, prefrontal hemodynamics, and multimodal, from eye-tracking and neural hemodynamics, functional near-infrared spectroscopy, (fNIRS) windows using attention-based encoders, and combines them with differentiable approximate reasoning rules using learned weights and soft thresholds, to address both rigid hand-crafted rules and the lack of subject-level alignment diagnostics. We apply this system to fatigue classification from multimodal physiological signals, a domain that requires models that are accurate and interpretable, with internal reasoning that can be inspected for safety-critical use. In leave-one-subject-out evaluation on 18 participants (560 samples), the method achieves 72.1% +/- 12.3% accuracy, comparable to tuned baselines while exposing concept activations and rule firing strengths. Ablations indicate gains from participant-specific calibration (+5.2 pp), a modest drop without the fNIRS concept (-1.2 pp), and slightly better performance with Lukasiewicz operators than product (+0.9 pp). We also introduce concept fidelity, an offline per-subject audit metric from held-out labels, which correlates strongly with per-subject accuracy (r=0.843, p < 0.0001).