R&D: Balancing Reliability and Diversity in Synthetic Data Augmentation for Semantic Segmentation

TL;DR

This paper introduces a novel synthetic data augmentation pipeline using controllable diffusion models to improve semantic segmentation by balancing diversity and reliability, especially in data-scarce scenarios.

Contribution

The work presents a new augmentation method that integrates class-aware prompting and visual prior blending with diffusion models for pixel-level tasks.

Findings

Significantly improves segmentation performance on PASCAL VOC and BDD100K datasets.

Enhances model robustness in real-world applications.

Effective in data-scarce scenarios.

Abstract

Collecting and annotating datasets for pixel-level semantic segmentation tasks are highly labor-intensive. Data augmentation provides a viable solution by enhancing model generalization without additional real-world data collection. Traditional augmentation techniques, such as translation, scaling, and color transformations, create geometric variations but fail to generate new structures. While generative models have been employed to extend semantic information of datasets, they often struggle to maintain consistency between the original and generated images, particularly for pixel-level tasks. In this work, we propose a novel synthetic data augmentation pipeline that integrates controllable diffusion models. Our approach balances diversity and reliability data, effectively bridging the gap between synthetic and real data. We utilize class-aware prompting and visual prior blending to improve image quality further, ensuring precise alignment with segmentation labels. By evaluating benchmark datasets such as PASCAL VOC and BDD100K, we demonstrate that our method significantly enhances semantic segmentation performance, especially in data-scarce scenarios, while improving model robustness in real-world applications. Our code is available at \href{https://github.com/chequanghuy/Enhanced-Generative-Data-Augmentation-for-Semantic-Segmentation-via-Stronger-Guidance}{https://github.com/chequanghuy/Enhanced-Generative-Data-Augmentation-for-Semantic-Segmentation-via-Stronger-Guidance}.