Decoupled Audio-Visual Dataset Distillation

TL;DR

This paper introduces DAVDD, a decoupled audio-visual dataset distillation framework that uses pretrained models and disentangled representations to better preserve cross-modal structure and modality-specific information, achieving state-of-the-art results.

Contribution

The paper proposes a novel decoupled distillation framework leveraging pretrained banks and disentangled representations to improve audio-visual dataset compression.

Findings

Achieves state-of-the-art results across multiple benchmarks.

Effectively preserves cross-modal structure and modality-specific cues.

Demonstrates robustness under various IPC settings.

Abstract

Audio-Visual Dataset Distillation aims to compress large-scale datasets into compact subsets while preserving the performance of the original data. However, conventional Distribution Matching (DM) methods struggle to capture intrinsic cross-modal alignment. Subsequent studies have attempted to introduce cross-modal matching, but two major challenges remain: (i) independently and randomly initialized encoders lead to inconsistent modality mapping spaces, increasing training difficulty; and (ii) direct interactions between modalities tend to damage modality-specific (private) information, thereby degrading the quality of the distilled data. To address these challenges, we propose DAVDD, a pretraining-based decoupled audio-visual distillation framework. DAVDD leverages a diverse pretrained bank to obtain stable modality features and uses a lightweight decoupler bank to disentangle them into common and private representations. To effectively preserve cross-modal structure, we further introduce Common Intermodal Matching together with a Sample-Distribution Joint Alignment strategy, ensuring that shared representations are aligned both at the sample level and the global distribution level. Meanwhile, private representations are entirely isolated from cross-modal interaction, safeguarding modality-specific cues throughout distillation. Extensive experiments across multiple benchmarks show that DAVDD achieves state-of-the-art results under all IPC settings, demonstrating the effectiveness of decoupled representation learning for high-quality audio-visual dataset distillation. Code will be released.