Adaptive Dataset Quantization: A New Direction for Dataset Pruning

TL;DR

This paper introduces a novel dataset quantization method that compresses individual samples within datasets using adaptive ratios, significantly reducing storage costs while preserving model performance, especially suitable for resource-limited edge devices.

Contribution

It is the first to utilize limited bits for dataset storage reduction through an adaptive quantization algorithm at the dataset level, improving compression efficiency over traditional methods.

Findings

Maintains model training performance after compression

Achieves significant dataset size reduction

Outperforms traditional quantization and pruning baselines

Abstract

This paper addresses the challenges of storage and communication costs for large-scale datasets in resource-constrained edge devices by proposing a novel dataset quantization approach to reduce intra-sample redundancy. Unlike traditional dataset pruning and distillation methods that focus on inter-sample redundancy, the proposed method compresses each image by reducing redundant or less informative content within samples while preserving essential features. It first applies linear symmetric quantization to obtain an initial quantization range and scale for each sample. Then, an adaptive quantization allocation algorithm is introduced to distribute different quantization ratios for samples with varying precision requirements, maintaining a constant total compression ratio. The main contributions include: (1) being the first to use limited bits to represent datasets for storage reduction; (2) introducing a dataset-level quantization algorithm with adaptive ratio allocation; and (3) validating the method's effectiveness through extensive experiments on CIFAR-10, CIFAR-100, and ImageNet-1K. Results show that the method maintains model training performance while achieving significant dataset compression, outperforming traditional quantization and dataset pruning baselines under the same compression ratios.