Thermal to Visible Image Synthesis under Atmospheric Turbulence

TL;DR

This paper introduces a novel end-to-end method for converting thermal images into visible-spectrum images under atmospheric turbulence, improving image quality and face verification accuracy in long-range imaging applications.

Contribution

First to study thermal to visible image translation under atmospheric turbulence, proposing an end-to-end reconstruction method utilizing a pre-trained StyleGAN2 network.

Findings

Effective in improving visual quality of reconstructed images

Enhances face verification accuracy under turbulence conditions

Outperforms existing two-step methods

Abstract

In many practical applications of long-range imaging such as biometrics and surveillance, thermal imagining modalities are often used to capture images in low-light and nighttime conditions. However, such imaging systems often suffer from atmospheric turbulence, which introduces severe blur and deformation artifacts to the captured images. Such an issue is unavoidable in long-range imaging and significantly decreases the face verification accuracy. In this paper, we first investigate the problem with a turbulence simulation method on real-world thermal images. An end-to-end reconstruction method is then proposed which can directly transform thermal images into visible-spectrum images by utilizing natural image priors based on a pre-trained StyleGAN2 network. Compared with the existing two-steps methods of consecutive turbulence mitigation and thermal to visible image translation, our method is demonstrated to be effective in terms of both the visual quality of the reconstructed results and face verification accuracy. Moreover, to the best of our knowledge, this is the first work that studies the problem of thermal to visible image translation under atmospheric turbulence.