BTSeg: Barlow Twins Regularization for Domain Adaptation in Semantic Segmentation

TL;DR

BTSeg introduces a semi-supervised regularization method using Barlow Twins to improve semantic segmentation under adverse weather, achieving state-of-the-art results without extra labeled data.

Contribution

The paper presents BTSeg, a novel semi-supervised regularization approach leveraging Barlow Twins for domain adaptation in semantic segmentation.

Findings

Achieves top performance on ACG benchmark.

Sets new state-of-the-art on ACDC dataset.

Effective in adverse weather conditions without additional labels.

Abstract

We introduce BTSeg (Barlow Twins regularized Segmentation), an innovative, semi-supervised training approach enhancing semantic segmentation models in order to effectively tackle adverse weather conditions without requiring additional labeled training data. Images captured at similar locations but under varying adverse conditions are regarded as manifold representation of the same scene, thereby enabling the model to conceptualize its understanding of the environment. BTSeg shows cutting-edge performance for the new challenging ACG benchmark and sets a new state-of-the-art for weakly-supervised domain adaptation for the ACDC dataset. To support further research, we have made our code publicly available at https://github.com/fraunhoferhhi/BTSeg .