Text2Traffic: A Text-to-Image Generation and Editing Method for Traffic Scenes

TL;DR

Text2Traffic is a unified framework that improves text-driven traffic scene image generation and editing by enhancing semantic richness, visual fidelity, and multi-view diversity through a controllable mask mechanism and a two-stage training process.

Contribution

It introduces a novel controllable mask mechanism and a two-stage training paradigm for improved traffic scene image synthesis and editing from text descriptions.

Findings

Achieves state-of-the-art performance in traffic scene image generation.

Enhances small traffic element fidelity with mask-region-weighted loss.

Increases geometric diversity using multi-view data.

Abstract

With the rapid advancement of intelligent transportation systems, text-driven image generation and editing techniques have demonstrated significant potential in providing rich, controllable visual scene data for applications such as traffic monitoring and autonomous driving. However, several challenges remain, including insufficient semantic richness of generated traffic elements, limited camera viewpoints, low visual fidelity of synthesized images, and poor alignment between textual descriptions and generated content. To address these issues, we propose a unified text-driven framework for both image generation and editing, leveraging a controllable mask mechanism to seamlessly integrate the two tasks. Furthermore, we incorporate both vehicle-side and roadside multi-view data to enhance the geometric diversity of traffic scenes. Our training strategy follows a two-stage paradigm: first, we perform conceptual learning using large-scale coarse-grained text-image data; then, we fine-tune with fine-grained descriptive data to enhance text-image alignment and detail quality. Additionally, we introduce a mask-region-weighted loss that dynamically emphasizes small yet critical regions during training, thereby substantially enhancing the generation fidelity of small-scale traffic elements. Extensive experiments demonstrate that our method achieves leading performance in text-based image generation and editing within traffic scenes.