SGD-Mix: Enhancing Domain-Specific Image Classification with Label-Preserving Data Augmentation

TL;DR

This paper introduces SGD-Mix, a novel data augmentation framework for domain-specific image classification that combines saliency-guided mixing and diffusion models to improve diversity, faithfulness, and label clarity, leading to better downstream performance.

Contribution

The paper presents a new augmentation method integrating saliency-guided mixing with diffusion models to address key challenges in domain-specific image classification.

Findings

Outperforms state-of-the-art augmentation methods across various tasks.

Enhances foreground semantics and background diversity effectively.

Improves robustness in fine-grained, long-tail, and few-shot scenarios.

Abstract

Data augmentation for domain-specific image classification tasks often struggles to simultaneously address diversity, faithfulness, and label clarity of generated data, leading to suboptimal performance in downstream tasks. While existing generative diffusion model-based methods aim to enhance augmentation, they fail to cohesively tackle these three critical aspects and often overlook intrinsic challenges of diffusion models, such as sensitivity to model characteristics and stochasticity under strong transformations. In this paper, we propose a novel framework that explicitly integrates diversity, faithfulness, and label clarity into the augmentation process. Our approach employs saliency-guided mixing and a fine-tuned diffusion model to preserve foreground semantics, enrich background diversity, and ensure label consistency, while mitigating diffusion model limitations. Extensive experiments across fine-grained, long-tail, few-shot, and background robustness tasks demonstrate our method's superior performance over state-of-the-art approaches.