Similarity Min-Max: Zero-Shot Day-Night Domain Adaptation

TL;DR

This paper introduces a zero-shot day-night domain adaptation method that jointly optimizes image-level darkening and model-level feature similarity to improve performance across various nighttime vision tasks without using nighttime training data.

Contribution

It proposes a novel similarity min-max paradigm that unifies image-level and model-level adaptation for zero-shot day-night domain transfer, a first in the field.

Findings

Significant improvements in nighttime vision tasks.

Effective across classification, segmentation, recognition, and video tasks.

Broad applicability demonstrated through extensive experiments.

Abstract

Low-light conditions not only hamper human visual experience but also degrade the model's performance on downstream vision tasks. While existing works make remarkable progress on day-night domain adaptation, they rely heavily on domain knowledge derived from the task-specific nighttime dataset. This paper challenges a more complicated scenario with border applicability, i.e., zero-shot day-night domain adaptation, which eliminates reliance on any nighttime data. Unlike prior zero-shot adaptation approaches emphasizing either image-level translation or model-level adaptation, we propose a similarity min-max paradigm that considers them under a unified framework. On the image level, we darken images towards minimum feature similarity to enlarge the domain gap. Then on the model level, we maximize the feature similarity between the darkened images and their normal-light counterparts for better model adaptation. To the best of our knowledge, this work represents the pioneering effort in jointly optimizing both aspects, resulting in a significant improvement of model generalizability. Extensive experiments demonstrate our method's effectiveness and broad applicability on various nighttime vision tasks, including classification, semantic segmentation, visual place recognition, and video action recognition. Code and pre-trained models are available at https://red-fairy.github.io/ZeroShotDayNightDA-Webpage/.