Inversion Circle Interpolation: Diffusion-based Image Augmentation for Data-scarce Classification

TL;DR

This paper introduces Diff-II, a diffusion-based data augmentation method that enhances image classification in data-scarce scenarios by combining category concept learning, inversion interpolation, and two-stage denoising.

Contribution

The paper proposes a novel diffusion-based data augmentation technique, Diff-II, which balances faithfulness and diversity to improve classification performance in limited data settings.

Findings

Outperforms state-of-the-art diffusion-based DA methods.

Effective in few-shot, long-tailed, and out-of-distribution classification.

Demonstrates significant accuracy improvements across various tasks.

Abstract

Data Augmentation (DA), i.e., synthesizing faithful and diverse samples to expand the original training set, is a prevalent and effective strategy to improve the performance of various data-scarce tasks. With the powerful image generation ability, diffusion-based DA has shown strong performance gains on different image classification benchmarks. In this paper, we analyze today's diffusion-based DA methods, and argue that they cannot take account of both faithfulness and diversity, which are two critical keys for generating high-quality samples and boosting classification performance. To this end, we propose a novel Diffusion-based DA method: Diff-II. Specifically, it consists of three steps: 1) Category concepts learning: Learning concept embeddings for each category. 2) Inversion interpolation: Calculating the inversion for each image, and conducting circle interpolation for two randomly sampled inversions from the same category. 3) Two-stage denoising: Using different prompts to generate synthesized images in a coarse-to-fine manner. Extensive experiments on various data-scarce image classification tasks (e.g., few-shot, long-tailed, and out-of-distribution classification) have demonstrated its effectiveness over state-of-the-art diffusion-based DA methods.