Mitigating Long-Tail Bias via Prompt-Controlled Diffusion Augmentation

TL;DR

This paper introduces a prompt-controlled diffusion augmentation method that generates targeted synthetic data to address class imbalance and domain shifts in remote-sensing image segmentation.

Contribution

It presents a novel diffusion-based framework for controlled, targeted data augmentation that improves segmentation of minority classes across domains.

Findings

Synthetic data improves segmentation accuracy, especially for minority classes.

The method enhances performance under domain shift conditions.

Controlled augmentation outperforms indiscriminate dataset expansion.

Abstract

Long-tailed class imbalance remains a fundamental obstacle in semantic segmentation of high-resolution remote-sensing imagery, where dominant classes shape learned representations and rare classes are systematically under-segmented. This challenge becomes more acute in cross-domain settings such as LoveDA, which exhibits an explicit Urban/Rural split with substantial appearance differences and inconsistent class-frequency statistics across domains. We propose a prompt-controlled diffusion augmentation framework that generates paired label-image samples with explicit control over semantic composition and domain, enabling targeted enrichment of underrepresented classes rather than indiscriminate dataset expansion. A domain-aware, masked, ratio-conditioned discrete diffusion model first synthesizes layouts that satisfy class-ratio targets while preserving realistic spatial co-occurrence, and a ControlNet-guided diffusion model then renders photorealistic, domain-consistent images from these layouts. When mixed with real data, the resulting synthetic pairs improve multiple segmentation backbones, especially on minority classes and under domain shift, showing that better downstream segmentation comes from adding the right samples in the right proportions. Source codes, pretrained models, and synthetic datasets are available at \href{https://buddhi19.github.io/SyntheticGen}{\texttt{buddhi19.github.io/SyntheticGen}}.