SAS: Semantic-aware Sampling for Generative Dataset Distillation

TL;DR

This paper introduces SAS, a semantic-aware sampling method for dataset distillation that leverages CLIP to produce compact, semantically rich datasets, improving downstream model performance.

Contribution

It proposes a novel semantic scoring and sampling strategy using CLIP to enhance dataset distillation with semantic relevance and diversity.

Findings

Consistent performance improvements across multiple datasets.

Effective filtering of semantically relevant samples.

Enhanced semantic class discrimination in distilled datasets.

Abstract

Deep neural networks have achieved impressive performance across a wide range of tasks, but this success often comes with substantial computational and storage costs due to large-scale training data. Dataset distillation addresses this challenge by constructing compact yet informative datasets that enable efficient model training while maintaining downstream performance. However, most existing approaches primarily emphasize matching data distributions or downstream training statistics, with limited attention to preserving high-level semantic information in the distilled data. In this work, we introduce a semantic-aware perspective for dataset distillation by leveraging Contrastive Language-Image Pretraining (CLIP) as a semantic prior for post-sampling. Our goal is to obtain distilled datasets that are not only compact but also semantically class-discriminative and diverse. To this end, we design three semantic scoring functions that quantify class relevance, inter-class separability, and intra-set diversity in a pretrained semantic space. Based on image pools generated by existing distillation methods, we further develop a two-stage strategy for effective sampling: the first stage filters semantically discriminative samples to form a reliable candidate set, and the second stage performs a dynamic diversity-aware selection to reduce redundancy while preserving semantic coverage. Extensive experiments across multiple datasets, image pools, and downstream models demonstrate consistent performance gains, highlighting the effectiveness of incorporating semantic information into dataset distillation.