Color-Oriented Redundancy Reduction in Dataset Distillation

TL;DR

This paper introduces AutoPalette, a novel framework for dataset distillation that reduces color redundancy in synthetic images by dynamically allocating colors and selecting diverse representative images, improving training efficiency.

Contribution

AutoPalette is the first method to explicitly minimize color redundancy at both image and dataset levels in dataset distillation, enhancing the quality of condensed datasets.

Findings

AutoPalette outperforms existing dataset distillation methods across multiple datasets.

The palette network effectively allocates essential colors to synthetic images.

Color-guided initialization reduces redundancy and improves dataset diversity.

Abstract

Dataset Distillation (DD) is designed to generate condensed representations of extensive image datasets, enhancing training efficiency. Despite recent advances, there remains considerable potential for improvement, particularly in addressing the notable redundancy within the color space of distilled images. In this paper, we propose AutoPalette, a framework that minimizes color redundancy at the individual image and overall dataset levels, respectively. At the image level, we employ a palette network, a specialized neural network, to dynamically allocate colors from a reduced color space to each pixel. The palette network identifies essential areas in synthetic images for model training and consequently assigns more unique colors to them. At the dataset level, we develop a color-guided initialization strategy to minimize redundancy among images. Representative images with the least replicated color patterns are selected based on the information gain. A comprehensive performance study involving various datasets and evaluation scenarios is conducted, demonstrating the superior performance of our proposed color-aware DD compared to existing DD methods. The code is available at \url{https://github.com/KeViNYuAn0314/AutoPalette}.