A Novel Deep Learning Method for Thermal to Annotated Thermal-Optical Fused Images

TL;DR

This paper introduces a pioneering deep learning approach for thermal to optical image fusion using Discrete Wavelet Transform, creating a new database and a statistical measure for region of interest detection, improving thermal image analysis.

Contribution

The work is the first to perform thermal-optical grayscale fusion using deep learning in the DWT domain, along with a new database and a statistical ROI detection method.

Findings

Encouraging results in ROI identification in fused images

Better processing of fused images in mixed form than thermal alone

Introduction of a new thermal-optical fusion database

Abstract

Thermal Images profile the passive radiation of objects and capture them in grayscale images. Such images have a very different distribution of data compared to optical colored images. We present here a work that produces a grayscale thermo-optical fused mask given a thermal input. This is a deep learning based pioneering work since to the best of our knowledge, there exists no other work on thermal-optical grayscale fusion. Our method is also unique in the sense that the deep learning method we are proposing here works on the Discrete Wavelet Transform (DWT) domain instead of the gray level domain. As a part of this work, we also present a new and unique database for obtaining the region of interest in thermal images based on an existing thermal visual paired database, containing the Region of Interest on 5 different classes of data. Finally, we are proposing a simple low cost overhead statistical measure for identifying the region of interest in the fused images, which we call as the Region of Fusion (RoF). Experiments on the database show encouraging results in identifying the region of interest in the fused images. We also show that they can be processed better in the mixed form rather than with only thermal images.