Clinical-Prior Guided Multi-Modal Learning with Latent Attention Pooling for Gait-Based Scoliosis Screening

TL;DR

This paper introduces ScoliGait, a new gait video dataset and a multi-modal learning framework that leverages clinical priors and latent attention pooling to improve non-invasive scoliosis screening accuracy.

Contribution

The paper presents a novel dataset and a multi-modal model integrating clinical priors with video and text data for interpretable scoliosis screening.

Findings

Achieved state-of-the-art performance on a non-repeating subject benchmark.

Demonstrated the effectiveness of clinical-prior-guided knowledge maps.

Established a scalable, interpretable framework for gait-based scoliosis assessment.

Abstract

Adolescent Idiopathic Scoliosis (AIS) is a prevalent spinal deformity whose progression can be mitigated through early detection. Conventional screening methods are often subjective, difficult to scale, and reliant on specialized clinical expertise. Video-based gait analysis offers a promising alternative, but current datasets and methods frequently suffer from data leakage, where performance is inflated by repeated clips from the same individual, or employ oversimplified models that lack clinical interpretability. To address these limitations, we introduce ScoliGait, a new benchmark dataset comprising 1,572 gait video clips for training and 300 fully independent clips for testing. Each clip is annotated with radiographic Cobb angles and descriptive text based on clinical kinematic priors. We propose a multi-modal framework that integrates a clinical-prior-guided kinematic knowledge map for interpretable feature representation, alongside a latent attention pooling mechanism to fuse video, text, and knowledge map modalities. Our method establishes a new state-of-the-art, demonstrating a significant performance gap on a realistic, non-repeating subject benchmark. Our approach establishes a new state of the art, showing a significant performance gain on a realistic, subject-independent benchmark. This work provides a robust, interpretable, and clinically grounded foundation for scalable, non-invasive AIS assessment.