PIDNet: Progressive Implicit Decouple Network for Multimodal Action Quality Assessment

TL;DR

PIDNet introduces a progressive decoupling and fusion approach for multimodal action quality assessment, effectively integrating modality-specific cues and global semantics for improved accuracy.

Contribution

The paper proposes a novel PIDNet architecture with iMambaWave and Group3M modules for better multimodal feature disentanglement and fusion in action quality assessment.

Findings

PIDNet achieves state-of-the-art correlation scores on Rhythmic Gymnastics and Fis-V datasets.

The iMambaWave module enhances temporal and frequency domain representations.

Ablation studies confirm the effectiveness of each component in PIDNet.

Abstract

Action quality assessment (AQA) aims to automatically quantify the execution quality of human actions in videos and is valuable for applications such as competitive sports judging. In multimodal AQA, quality evidence from different modalities is heterogeneous, and quality cues evolve progressively over time. Existing methods often rely on coarse fusion or unified temporal modeling, which may blur modality-specific cues, preserve cross-modal redundancy, and weaken stage-specific quality evidence. To address these issues, we propose a progressive implicit decoupling and fusion network (PIDNet) that progressively integrates modality-specific information, cross-modal complementary cues, and global quality semantics for accurate assessment. Specifically, we design an iMambaWave module that maps RGB, optical flow, and audio features into a shared latent space and disentangles them with a Bi-Mamba branch and a wavelet-transform branch to capture long-range temporal dependencies and local perturbation details, respectively. A gated aggregation mechanism adaptively fuses temporal and frequency-domain information. We further build a three-stage progressive fusion network using Group3M blocks, where modality complementary attention retrieves cross-modal evidence while suppressing redundancy, and multi-scale convolutions enrich feature representations. Experiments on the Rhythmic Gymnastics and Fis-V datasets show that PIDNet achieves highly competitive score correlation with favorable error control compared with existing unimodal and multimodal methods. Ablation studies verify the effectiveness of each component. Moreover, iMambaWave consistently improves visual representation and temporal modeling across multiple backbones, showing good generalization and plug-and-play capability.