Exploring Probabilistic Modeling Beyond Domain Generalization for Semantic Segmentation

TL;DR

This paper introduces PDAF, a probabilistic diffusion framework that models latent domain priors to improve the generalization of semantic segmentation models across unseen environments, addressing limitations of feature alignment methods.

Contribution

We propose PDAF, a novel probabilistic diffusion alignment framework that captures latent domain priors and enhances domain generalization in semantic segmentation.

Findings

PDAF significantly improves segmentation accuracy on unseen domains.

The framework effectively models complex domain shifts in urban scenes.

Extensive experiments demonstrate PDAF's superior generalization performance.

Abstract

Domain Generalized Semantic Segmentation (DGSS) is a critical yet challenging task, as domain shifts in unseen environments can severely compromise model performance. While recent studies enhance feature alignment by projecting features into the source domain, they often neglect intrinsic latent domain priors, leading to suboptimal results. In this paper, we introduce PDAF, a Probabilistic Diffusion Alignment Framework that enhances the generalization of existing segmentation networks through probabilistic diffusion modeling. PDAF introduces a Latent Domain Prior (LDP) to capture domain shifts and uses this prior as a conditioning factor to align both source and unseen target domains. To achieve this, PDAF integrates into a pre-trained segmentation model and utilizes paired source and pseudo-target images to simulate latent domain shifts, enabling LDP modeling. The framework comprises three modules: the Latent Prior Extractor (LPE) predicts the LDP by supervising domain shifts; the Domain Compensation Module (DCM) adjusts feature representations to mitigate domain shifts; and the Diffusion Prior Estimator (DPE) leverages a diffusion process to estimate the LDP without requiring paired samples. This design enables PDAF to iteratively model domain shifts, progressively refining feature representations to enhance generalization under complex target conditions. Extensive experiments validate the effectiveness of PDAF across diverse and challenging urban scenes.