Dataset Distillation Using Parameter Pruning

TL;DR

This paper introduces a new dataset distillation technique that employs parameter pruning to create more robust datasets and enhance distillation effectiveness, validated through experiments on benchmark datasets.

Contribution

The paper presents a novel dataset distillation approach using parameter pruning to improve robustness and performance, which is a new direction in dataset synthesis methods.

Findings

Outperforms existing methods on benchmark datasets

Produces more robust distilled datasets

Improves distillation performance through parameter pruning

Abstract

In this study, we propose a novel dataset distillation method based on parameter pruning. The proposed method can synthesize more robust distilled datasets and improve distillation performance by pruning difficult-to-match parameters during the distillation process. Experimental results on two benchmark datasets show the superiority of the proposed method.