ThermalDiffusion: Visual-to-Thermal Image-to-Image Translation for Autonomous Navigation

TL;DR

This paper introduces ThermalDiffusion, a method that uses conditional diffusion models to generate synthetic thermal images from RGB data, enhancing datasets for autonomous navigation in challenging environments.

Contribution

It presents a novel approach employing self-attention-based diffusion models to convert RGB images into thermal images, addressing data scarcity in thermal imaging for robotics.

Findings

Effective generation of realistic thermal images from RGB data.

Improved dataset diversity for thermal imaging in autonomous systems.

Potential to accelerate thermal camera adoption in robotics.

Abstract

Autonomous systems rely on sensors to estimate the environment around them. However, cameras, LiDARs, and RADARs have their own limitations. In nighttime or degraded environments such as fog, mist, or dust, thermal cameras can provide valuable information regarding the presence of objects of interest due to their heat signature. They make it easy to identify humans and vehicles that are usually at higher temperatures compared to their surroundings. In this paper, we focus on the adaptation of thermal cameras for robotics and automation, where the biggest hurdle is the lack of data. Several multi-modal datasets are available for driving robotics research in tasks such as scene segmentation, object detection, and depth estimation, which are the cornerstone of autonomous systems. However, they are found to be lacking in thermal imagery. Our paper proposes a solution to augment these datasets with synthetic thermal data to enable widespread and rapid adaptation of thermal cameras. We explore the use of conditional diffusion models to convert existing RGB images to thermal images using self-attention to learn the thermal properties of real-world objects.