Bridging Day and Night: Target-Class Hallucination Suppression in Unpaired Image Translation

TL;DR

This paper introduces a novel unpaired image translation framework that detects and suppresses target-class hallucinations, improving the realism and semantic accuracy of day-to-night translations, especially for critical objects like traffic signs.

Contribution

It proposes a dual-head discriminator with semantic segmentation and class-specific prototypes to detect and suppress hallucinations during translation, enhancing downstream task performance.

Findings

Outperforms existing methods qualitatively and quantitatively.

Improves mAP by 15.5% on BDD100K for day-to-night translation.

Achieves 31.7% gain for classes prone to hallucinations, such as traffic lights.

Abstract

Day-to-night unpaired image translation is important to downstream tasks but remains challenging due to large appearance shifts and the lack of direct pixel-level supervision. Existing methods often introduce semantic hallucinations, where objects from target classes such as traffic signs and vehicles, as well as man-made light effects, are incorrectly synthesized. These hallucinations significantly degrade downstream performance. We propose a novel framework that detects and suppresses hallucinations of target-class features during unpaired translation. To detect hallucination, we design a dual-head discriminator that additionally performs semantic segmentation to identify hallucinated content in background regions. To suppress these hallucinations, we introduce class-specific prototypes, constructed by aggregating features of annotated target-domain objects, which act as semantic anchors for each class. Built upon a Schrodinger Bridge-based translation model, our framework performs iterative refinement, where detected hallucination features are explicitly pushed away from class prototypes in feature space, thus preserving object semantics across the translation trajectory.Experiments show that our method outperforms existing approaches both qualitatively and quantitatively. On the BDD100K dataset, it improves mAP by 15.5% for day-to-night domain adaptation, with a notable 31.7% gain for classes such as traffic lights that are prone to hallucinations.