DAM: Domain-Aware Module for Multi-Domain Dataset Condensation

TL;DR

This paper introduces a domain-aware module for multi-domain dataset condensation, improving the generalization of condensed datasets across various domains and architectures by embedding domain features via learnable masks.

Contribution

We propose the Domain-Aware Module (DAM) with pseudo-domain labeling to enhance multi-domain dataset condensation, addressing the heterogeneity of modern datasets.

Findings

DAM improves in-domain performance.

DAM enhances out-of-domain generalization.

DAM boosts cross-architecture transferability.

Abstract

Dataset Condensation (DC) has emerged as a promising solution to mitigate the computational and storage burdens associated with training deep learning models. However, existing DC methods largely overlook the multi-domain nature of modern datasets, which are increasingly composed of heterogeneous images spanning multiple domains. In this paper, we extend DC and introduce Multi-Domain Dataset Condensation (MDDC), which aims to condense data that generalizes across both single-domain and multi-domain settings. To this end, we propose the Domain-Aware Module (DAM), a training-time module that embeds domain-related features into each synthetic image via learnable spatial masks. As explicit domain labels are mostly unavailable in real-world datasets, we employ frequency-based pseudo-domain labeling, which leverages low-frequency amplitude statistics. DAM is only active during the condensation process, thus preserving the same images per class (IPC) with prior methods. Experiments show that DAM consistently improves in-domain, out-of-domain, and cross-architecture performance over baseline dataset condensation methods.