Instance-Guided Unsupervised Domain Adaptation for Robotic Semantic Segmentation

TL;DR

This paper introduces an unsupervised domain adaptation method for robotic semantic segmentation that uses multi-view consistent pseudo-labels refined by foundation models, improving performance without ground-truth labels.

Contribution

It proposes a novel approach combining volumetric 3D maps and foundation model-based label refinement for effective unsupervised domain adaptation in robotic perception.

Findings

Outperforms state-of-the-art UDA methods on real-world data

Does not require ground-truth labels in the target domain

Enhances multi-view consistency and instance-level coherence

Abstract

Semantic segmentation networks, which are essential for robotic perception, often suffer from performance degradation when the visual distribution of the deployment environment differs from that of the source dataset on which they were trained. Unsupervised Domain Adaptation (UDA) addresses this challenge by adapting the network to the robot's target environment without external supervision, leveraging the large amounts of data a robot might naturally collect during long-term operation. In such settings, UDA methods can exploit multi-view consistency across the environment's map to fine-tune the model in an unsupervised fashion and mitigate domain shift. However, these approaches remain sensitive to cross-view instance-level inconsistencies. In this work, we propose a method that starts from a volumetric 3D map to generate multi-view consistent pseudo-labels. We then refine these labels using the zero-shot instance segmentation capabilities of a foundation model, enforcing instance-level coherence. The refined annotations serve as supervision for self-supervised fine-tuning, enabling the robot to adapt its perception system at deployment time. Experiments on real-world data demonstrate that our approach consistently improves performance over state-of-the-art UDA baselines based on multi-view consistency, without requiring any ground-truth labels in the target domain.