Dataset Augmentation by Mixing Visual Concepts

TL;DR

This paper introduces a novel dataset augmentation method using fine-tuned diffusion models with a technique called Mixing Visual Concepts, which generates diverse, realistic images to improve classification performance.

Contribution

The paper presents a new fine-tuning approach with Mixing Visual Concepts for generating diverse images, enhancing dataset augmentation effectiveness.

Findings

Outperforms existing augmentation methods on benchmark classification tasks.

Produces diverse images with both coarse and fine-grained variations.

Effective in reducing domain discrepancy between real and generated data.

Abstract

This paper proposes a dataset augmentation method by fine-tuning pre-trained diffusion models. Generating images using a pre-trained diffusion model with textual conditioning often results in domain discrepancy between real data and generated images. We propose a fine-tuning approach where we adapt the diffusion model by conditioning it with real images and novel text embeddings. We introduce a unique procedure called Mixing Visual Concepts (MVC) where we create novel text embeddings from image captions. The MVC enables us to generate multiple images which are diverse and yet similar to the real data enabling us to perform effective dataset augmentation. We perform comprehensive qualitative and quantitative evaluations with the proposed dataset augmentation approach showcasing both coarse-grained and finegrained changes in generated images. Our approach outperforms state-of-the-art augmentation techniques on benchmark classification tasks.