SGIA: Enhancing Fine-Grained Visual Classification with Sequence Generative Image Augmentation

TL;DR

This paper introduces SGIA, a novel data augmentation method using Sequence Latent Diffusion Models and Bridging Transfer Learning to improve fine-grained visual classification, especially in few-shot scenarios, by generating more realistic and diverse images.

Contribution

The study presents a new generative augmentation approach with BTL that enhances dataset variability and realism, outperforming existing methods in FGVC tasks.

Findings

Outperforms existing augmentation methods in FGVC accuracy

Generates more realistic and diverse images with pose variations

Achieves a 0.5% accuracy improvement on CUB-200-2011

Abstract

In Fine-Grained Visual Classification (FGVC), distinguishing highly similar subcategories remains a formidable challenge, often necessitating datasets with extensive variability. The acquisition and annotation of such FGVC datasets are notably difficult and costly, demanding specialized knowledge to identify subtle distinctions among closely related categories. Our study introduces a novel approach employing the Sequence Latent Diffusion Model (SLDM) for augmenting FGVC datasets, called Sequence Generative Image Augmentation (SGIA). Our method features a unique Bridging Transfer Learning (BTL) process, designed to minimize the domain gap between real and synthetically augmented data. This approach notably surpasses existing methods in generating more realistic image samples, providing a diverse range of pose transformations that extend beyond the traditional rigid transformations and style changes in generative augmentation. We demonstrate the effectiveness of our augmented dataset with substantial improvements in FGVC tasks on various datasets, models, and training strategies, especially in few-shot learning scenarios. Our method outperforms conventional image augmentation techniques in benchmark tests on three FGVC datasets, showcasing superior realism, variability, and representational quality. Our work sets a new benchmark and outperforms the previous state-of-the-art models in classification accuracy by 0.5% for the CUB-200-2011 dataset and advances the application of generative models in FGVC data augmentation.