Exploring Reliable Matching with Phase Enhancement for Night-time Semantic Segmentation

TL;DR

This paper introduces NightFormer, an end-to-end method for night-time semantic segmentation that enhances texture and reliable matching to improve accuracy in low-light conditions, outperforming existing approaches.

Contribution

The paper presents a novel night-time segmentation approach with pixel-level texture enhancement and object-level reliable matching modules, avoiding the need to adapt day-time models to night conditions.

Findings

Outperforms state-of-the-art methods on NightCity, BDD, and Cityscapes benchmarks.

Effectively enhances texture features for low-light environments.

Achieves more accurate object association in night-time images.

Abstract

Semantic segmentation of night-time images holds significant importance in computer vision, particularly for applications like night environment perception in autonomous driving systems. However, existing methods tend to parse night-time images from a day-time perspective, leaving the inherent challenges in low-light conditions (such as compromised texture and deceiving matching errors) unexplored. To address these issues, we propose a novel end-to-end optimized approach, named NightFormer, tailored for night-time semantic segmentation, avoiding the conventional practice of forcibly fitting night-time images into day-time distributions. Specifically, we design a pixel-level texture enhancement module to acquire texture-aware features hierarchically with phase enhancement and amplified attention, and an object-level reliable matching module to realize accurate association matching via reliable attention in low-light environments. Extensive experimental results on various challenging benchmarks including NightCity, BDD and Cityscapes demonstrate that our proposed method performs favorably against state-of-the-art night-time semantic segmentation methods.