HiGFA: Hierarchical Guidance for Fine-grained Data Augmentation with Diffusion Models

TL;DR

HiGFA introduces a hierarchical guidance method for diffusion models that improves fine-grained data augmentation by balancing global structure and detailed features, leading to more accurate synthetic images for fine-grained classification.

Contribution

It proposes a novel hierarchical guidance framework that dynamically modulates guidance signals during diffusion sampling for enhanced fine-grained image synthesis.

Findings

Improves synthetic image fidelity for fine-grained tasks

Enhances classifier performance with augmented data

Demonstrates effectiveness across multiple FGVC datasets

Abstract

Generative diffusion models show promise for data augmentation. However, applying them to fine-grained tasks presents a significant challenge: ensuring synthetic images accurately capture the subtle, category-defining features critical for high fidelity. Standard approaches, such as text-based Classifier-Free Guidance (CFG), often lack the required specificity, potentially generating misleading examples that degrade fine-grained classifier performance. To address this, we propose Hierarchically Guided Fine-grained Augmentation (HiGFA). HiGFA leverages the temporal dynamics of the diffusion sampling process. It employs strong text and transformed contour guidance with fixed strengths in the early-to-mid sampling stages to establish overall scene, style, and structure. In the final sampling stages, HiGFA activates a specialized fine-grained classifier guidance and dynamically modulates the strength of all guidance signals based on prediction confidence. This hierarchical, confidence-driven orchestration enables HiGFA to generate diverse yet faithful synthetic images by intelligently balancing global structure formation with precise detail refinement. Experiments on several FGVC datasets demonstrate the effectiveness of HiGFA.