Disentangled Contrastive Image Translation for Nighttime Surveillance

TL;DR

This paper introduces a novel disentangled contrastive learning approach for night-to-day image translation in surveillance, utilizing a physical prior and a new dataset to improve semantic consistency and translation quality in challenging nighttime conditions.

Contribution

It proposes a Disentangled Contrastive learning framework with a physical prior and a new surveillance dataset, advancing nighttime scene translation without supervision.

Findings

Outperforms existing nighttime translation methods significantly

Achieves high-fidelity, instance-aware translation in complex lighting conditions

Introduces a new dataset supporting nighttime surveillance research

Abstract

Nighttime surveillance suffers from degradation due to poor illumination and arduous human annotations. It is challengable and remains a security risk at night. Existing methods rely on multi-spectral images to perceive objects in the dark, which are troubled by low resolution and color absence. We argue that the ultimate solution for nighttime surveillance is night-to-day translation, or Night2Day, which aims to translate a surveillance scene from nighttime to the daytime while maintaining semantic consistency. To achieve this, this paper presents a Disentangled Contrastive (DiCo) learning method. Specifically, to address the poor and complex illumination in the nighttime scenes, we propose a learnable physical prior, i.e., the color invariant, which provides a stable perception of a highly dynamic night environment and can be incorporated into the learning pipeline of neural networks. Targeting the surveillance scenes, we develop a disentangled representation, which is an auxiliary pretext task that separates surveillance scenes into the foreground and background with contrastive learning. Such a strategy can extract the semantics without supervision and boost our model to achieve instance-aware translation. Finally, we incorporate all the modules above into generative adversarial networks and achieve high-fidelity translation. This paper also contributes a new surveillance dataset called NightSuR. It includes six scenes to support the study on nighttime surveillance. This dataset collects nighttime images with different properties of nighttime environments, such as flare and extreme darkness. Extensive experiments demonstrate that our method outperforms existing works significantly. The dataset and source code will be released on GitHub soon.