On the Road to Online Adaptation for Semantic Image Segmentation

TL;DR

This paper introduces a new online adaptation protocol for semantic image segmentation, enabling models to continuously learn from sequential data without supervision, aiming to better reflect real-world dynamic environments.

Contribution

It proposes a novel online learning framework and evaluation protocol for unsupervised domain adaptation in semantic segmentation, addressing limitations of previous offline and multi-domain approaches.

Findings

Baseline models demonstrate varying adaptation capabilities in the new protocol.

Analysis reveals challenges and potential directions for online unsupervised adaptation.

The framework sets a foundation for future research in continuous, real-time learning systems.

Abstract

We propose a new problem formulation and a corresponding evaluation framework to advance research on unsupervised domain adaptation for semantic image segmentation. The overall goal is fostering the development of adaptive learning systems that will continuously learn, without supervision, in ever-changing environments. Typical protocols that study adaptation algorithms for segmentation models are limited to few domains, adaptation happens offline, and human intervention is generally required, at least to annotate data for hyper-parameter tuning. We argue that such constraints are incompatible with algorithms that can continuously adapt to different real-world situations. To address this, we propose a protocol where models need to learn online, from sequences of temporally correlated images, requiring continuous, frame-by-frame adaptation. We accompany this new protocol with a variety of baselines to tackle the proposed formulation, as well as an extensive analysis of their behaviors, which can serve as a starting point for future research.