FREST: Feature RESToration for Semantic Segmentation under Multiple Adverse Conditions

TL;DR

FREST is a feature restoration framework for source-free domain adaptation in semantic segmentation, effectively improving performance under adverse conditions without requiring labeled normal images during training.

Contribution

It introduces a novel alternating training method that learns a condition embedding space and restores features, advancing SFDA for adverse condition segmentation.

Findings

Achieved state-of-the-art results on ACDC and RobotCar benchmarks.

Demonstrated superior generalization to unseen datasets.

Effectively reduces adverse condition effects in feature restoration.

Abstract

Robust semantic segmentation under adverse conditions is crucial in real-world applications. To address this challenging task in practical scenarios where labeled normal condition images are not accessible in training, we propose FREST, a novel feature restoration framework for source-free domain adaptation (SFDA) of semantic segmentation to adverse conditions. FREST alternates two steps: (1) learning the condition embedding space that only separates the condition information from the features and (2) restoring features of adverse condition images on the learned condition embedding space. By alternating these two steps, FREST gradually restores features where the effect of adverse conditions is reduced. FREST achieved a state of the art on two public benchmarks (i.e., ACDC and RobotCar) for SFDA to adverse conditions. Moreover, it shows superior generalization ability on unseen datasets.