DiRe: Diversity-promoting Regularization for Dataset Condensation

TL;DR

This paper introduces DiRe, a diversity regularizer for dataset condensation that reduces redundancy and enhances diversity, leading to improved generalization on benchmark datasets.

Contribution

The paper proposes a novel diversity regularizer that can be integrated into existing condensation methods to improve dataset diversity and utility.

Findings

Enhanced diversity in synthesized datasets

Improved generalization performance on benchmarks

Compatible with various condensation methods

Abstract

In Dataset Condensation, the goal is to synthesize a small dataset that replicates the training utility of a large original dataset. Existing condensation methods synthesize datasets with significant redundancy, so there is a dire need to reduce redundancy and improve the diversity of the synthesized datasets. To tackle this, we propose an intuitive Diversity Regularizer (DiRe) composed of cosine similarity and Euclidean distance, which can be applied off-the-shelf to various state-of-the-art condensation methods. Through extensive experiments, we demonstrate that the addition of our regularizer improves state-of-the-art condensation methods on various benchmark datasets from CIFAR-10 to ImageNet-1K with respect to generalization and diversity metrics.