Seeing Beyond: Extrapolative Domain Adaptive Panoramic Segmentation

TL;DR

This paper introduces EDA-PSeg, a novel framework for panoramic segmentation that effectively handles geometric distortions and open-set semantics, enabling robust 360-degree scene understanding across diverse domains.

Contribution

It proposes the EDA-PSeg framework with Euler-Margin Attention and Graph Matching Adapter to improve open-set domain adaptation for panoramic segmentation.

Findings

Achieves state-of-the-art results on four benchmark datasets.

Demonstrates robustness to geometric and environmental variations.

Effectively separates known and unknown classes in panoramic images.

Abstract

Cross-domain panoramic semantic segmentation has attracted growing interest as it enables comprehensive 360{\deg} scene understanding for real-world applications. However, it remains particularly challenging due to severe geometric Field of View (FoV) distortions and inconsistent open-set semantics across domains. In this work, we formulate an open-set domain adaptation setting, and propose Extrapolative Domain Adaptive Panoramic Segmentation (EDA-PSeg) framework that trains on local perspective views and tests on full 360{\deg} panoramic images, explicitly tackling both geometric FoV shifts across domains and semantic uncertainty arising from previously unseen classes. To this end, we propose the Euler-Margin Attention (EMA), which introduces an angular margin to enhance viewpoint-invariant semantic representation, while performing amplitude and phase modulation to improve generalization toward unseen classes. Additionally, we design the Graph Matching Adapter (GMA), which builds high-order graph relations to align shared semantics across FoV shifts while effectively separating novel categories through structural adaptation. Extensive experiments on four benchmark datasets under camera-shift, weather-condition, and open-set scenarios demonstrate that EDA-PSeg achieves state-of-the-art performance, robust generalization to diverse viewing geometries, and resilience under varying environmental conditions. The code is available at https://github.com/zyfone/EDA-PSeg.