IR2VI: Enhanced Night Environmental Perception by Unsupervised Thermal Image Translation

TL;DR

IR2VI is an unsupervised thermal-to-visible image translation framework that enhances night vision by learning from visual images and addressing common GAN challenges, improving environmental perception in dark conditions.

Contribution

The paper introduces IR2VI, a novel GAN-based unsupervised framework with structure connection and ROI focal loss to improve thermal-to-visible image translation for night vision.

Findings

IR2VI outperforms baseline methods in qualitative and quantitative evaluations.

The structure connection module improves detail preservation.

ROI focal loss enhances focus on regions of interest.

Abstract

Context enhancement is critical for night vision (NV) applications, especially for the dark night situation without any artificial lights. In this paper, we present the infrared-to-visual (IR2VI) algorithm, a novel unsupervised thermal-to-visible image translation framework based on generative adversarial networks (GANs). IR2VI is able to learn the intrinsic characteristics from VI images and integrate them into IR images. Since the existing unsupervised GAN-based image translation approaches face several challenges, such as incorrect mapping and lack of fine details, we propose a structure connection module and a region-of-interest (ROI) focal loss method to address the current limitations. Experimental results show the superiority of the IR2VI algorithm over baseline methods.