# Explainable AI for gait speed analysis from multimodal data fusion

## TL;DR

This paper presents a new AI framework that uses multiple data sources and deep learning to accurately analyze gait speed, with explainability through feature relevance analysis.

## Contribution

The novel contribution is a Multi-stream CNN architecture combined with LRP for explainable gait speed classification from multimodal data.

## Key findings

- The Multi-stream Quads CNN achieved highest F1 scores (96.6%–98%) across all experiments.
- LRP-based feature relevance analysis improved model robustness and interpretability.
- Multimodal fusion outperformed single-sensor approaches and traditional classifiers.

## Abstract

Gait speed analysis is vital for applications in healthcare, rehabilitation, human-robot interaction, and autonomous systems, necessitating robust methods to address its complexity. This study introduces an advanced framework for gait speed classification through multimodal data fusion and deep learning, optimized using Layer-wise Relevance Propagation (LRP). Utilizing a publicly available 4 datasets of 50 injury-free adults walking at varying speeds, we integrate data from full-body motion capture, electromyography (EMG), and force plates to capture detailed gait dynamics. We propose two novel architectures: a hybrid Convolutional Neural Network with Long Short-Term Memory (CNN+LSTM) and a Multi-stream CNN, benchmarked against Temporal Convolutional Networks (TCNs), Transformer neural networks, Gated Recurrent Units (GRU) and statistical classifiers (e.g., Linear Discriminant Analysis, Quadratic Discriminant Analysis, Support Vector Machines). All models in this study were trained, validated and tested using three distinct strategies: experiment 1 (sample-based splitting – 10-fold cross-validation), experiment 2 (10-fold cross-validation, subject-based splitting – 5 subjects for testing), and experiment 3 (5-fold cross-validation, subject-based splitting – 10 subjects for testing). Multi-stream Quads CNN is the best-performing model, achieving the highest F1 scores across all experiments (96.6% ± 1.4% experiment 1, 96.2% ± 1.6% experiment 2, 95.9% ± 0.9% experiment 3 with top performance for all experiments 98% ± 0.6%, outperforming traditional approaches. LRP is applied to evaluate feature relevance within the model, enabling the removal of critical features to verify robustness. Comprehensive assessments, including ROC curves, confusion matrices, t-test, and perturbation analyses, validate the model’s enhanced performance and durability. By combining multimodal fusion to overcome single-sensor constraints with LRP-driven feature optimization, this work provides a highly accurate and resilient tool for gait analysis, with significant potentials.