BriMA: Bridged Modality Adaptation for Multi-Modal Continual Action Quality Assessment

TL;DR

This paper introduces BriMA, a novel method for multi-modal Action Quality Assessment that effectively handles missing or intermittent modalities, improving robustness and accuracy in real-world scenarios.

Contribution

BriMA is the first approach to address modality-missing issues in continual multi-modal AQA, using memory-guided imputation and modality-aware replay mechanisms.

Findings

Achieves 6-8% higher correlation on benchmark datasets.

Reduces error rates by 12-15% under modality-missing conditions.

Demonstrates robustness in real-world deployment scenarios.

Abstract

Action Quality Assessment (AQA) aims to score how well an action is performed and is widely used in sports analysis, rehabilitation assessment, and human skill evaluation. Multi-modal AQA has recently achieved strong progress by leveraging complementary visual and kinematic cues, yet real-world deployments often suffer from non-stationary modality imbalance, where certain modalities become missing or intermittently available due to sensor failures or annotation gaps. Existing continual AQA methods overlook this issue and assume that all modalities remain complete and stable throughout training, which restricts their practicality. To address this challenge, we introduce Bridged Modality Adaptation (BriMA), an innovative approach to multi-modal continual AQA under modality-missing conditions. BriMA consists of a memory-guided bridging imputation module that reconstructs missing modalities using both task-agnostic and task-specific representations, and a modality-aware replay mechanism that prioritizes informative samples based on modality distortion and distribution drift. Experiments on three representative multi-modal AQA datasets (RG, Fis-V, and FS1000) show that BriMA consistently improves performance under different modality-missing conditions, achieving 6--8\% higher correlation and 12--15\% lower error on average. These results demonstrate a step toward robust multi-modal AQA systems under real-world deployment constraints.