Unlocking Constraints: Source-Free Occlusion-Aware Seamless Segmentation

TL;DR

This paper introduces UNLOCK, a novel source-free segmentation framework for panoramic images that handles occlusions and distortions, achieving state-of-the-art results without requiring source data during adaptation.

Contribution

It proposes the first solution to the source-free occlusion-aware panoramic segmentation task, with innovative modules for pseudo-labeling and context learning, advancing practical omni-view perception.

Findings

Achieves state-of-the-art performance with 10.9 mAAP and 11.6 mAP scores.

Demonstrates effective adaptation without source data in real and synthetic settings.

Improves mAPQ by +4.3 over source-only methods.

Abstract

Panoramic image processing is essential for omni-context perception, yet faces constraints like distortions, perspective occlusions, and limited annotations. Previous unsupervised domain adaptation methods transfer knowledge from labeled pinhole data to unlabeled panoramic images, but they require access to source pinhole data. To address these, we introduce a more practical task, i.e., Source-Free Occlusion-Aware Seamless Segmentation (SFOASS), and propose its first solution, called UNconstrained Learning Omni-Context Knowledge (UNLOCK). Specifically, UNLOCK includes two key modules: Omni Pseudo-Labeling Learning and Amodal-Driven Context Learning. While adapting without relying on source data or target labels, this framework enhances models to achieve segmentation with 360{\deg} viewpoint coverage and occlusion-aware reasoning. Furthermore, we benchmark the proposed SFOASS task through both real-to-real and synthetic-to-real adaptation settings. Experimental results show that our source-free method achieves performance comparable to source-dependent methods, yielding state-of-the-art scores of 10.9 in mAAP and 11.6 in mAP, along with an absolute improvement of +4.3 in mAPQ over the source-only method. All data and code will be made publicly available at https://github.com/yihong-97/UNLOCK.