LoopDA: Constructing Self-loops to Adapt Nighttime Semantic Segmentation

TL;DR

LoopDA introduces a novel self-loop framework with a co-teaching pipeline to improve nighttime semantic segmentation by reconstructing input data and aligning day-night features, significantly reducing the domain gap.

Contribution

The paper proposes LoopDA, a self-loop based approach with intra- and inter-domain refinement and a co-teaching pipeline for better domain adaptation in nighttime segmentation.

Findings

Outperforms prior methods on Dark Zurich dataset

Effective in reducing domain gap between day and night images

Enhances segmentation accuracy in adverse nighttime conditions

Abstract

Due to the lack of training labels and the difficulty of annotating, dealing with adverse driving conditions such as nighttime has posed a huge challenge to the perception system of autonomous vehicles. Therefore, adapting knowledge from a labelled daytime domain to an unlabelled nighttime domain has been widely researched. In addition to labelled daytime datasets, existing nighttime datasets usually provide nighttime images with corresponding daytime reference images captured at nearby locations for reference. The key challenge is to minimize the performance gap between the two domains. In this paper, we propose LoopDA for domain adaptive nighttime semantic segmentation. It consists of self-loops that result in reconstructing the input data using predicted semantic maps, by rendering them into the encoded features. In a warm-up training stage, the self-loops comprise of an inner-loop and an outer-loop, which are responsible for intra-domain refinement and inter-domain alignment, respectively. To reduce the impact of day-night pose shifts, in the later self-training stage, we propose a co-teaching pipeline that involves an offline pseudo-supervision signal and an online reference-guided signal `DNA' (Day-Night Agreement), bringing substantial benefits to enhance nighttime segmentation. Our model outperforms prior methods on Dark Zurich and Nighttime Driving datasets for semantic segmentation. Code and pretrained models are available at https://github.com/fy-vision/LoopDA.