GPS-GLASS: Learning Nighttime Semantic Segmentation Using Daytime Video and GPS data

TL;DR

This paper introduces a GPS-based training framework that enables nighttime semantic segmentation using only daytime images and GPS data, eliminating the need for annotated nighttime images.

Contribution

The proposed method leverages GPS-aligned daytime and nighttime images with cross-domain matching and flow estimation to generate pseudo supervision for training.

Findings

Effective nighttime segmentation without nighttime annotations

Improved performance on multiple datasets

Utilizes GPS data for cross-domain correspondence

Abstract

Semantic segmentation for autonomous driving should be robust against various in-the-wild environments. Nighttime semantic segmentation is especially challenging due to a lack of annotated nighttime images and a large domain gap from daytime images with sufficient annotation. In this paper, we propose a novel GPS-based training framework for nighttime semantic segmentation. Given GPS-aligned pairs of daytime and nighttime images, we perform cross-domain correspondence matching to obtain pixel-level pseudo supervision. Moreover, we conduct flow estimation between daytime video frames and apply GPS-based scaling to acquire another pixel-level pseudo supervision. Using these pseudo supervisions with a confidence map, we train a nighttime semantic segmentation network without any annotation from nighttime images. Experimental results demonstrate the effectiveness of the proposed method on several nighttime semantic segmentation datasets. Our source code is available at https://github.com/jimmy9704/GPS-GLASS.