Partially fake it till you make it: mixing real and fake thermal images for improved object detection

TL;DR

This paper introduces a novel data augmentation method that combines real and synthetic thermal images to enhance object detection performance in thermal videos, especially useful in data-scarce scenarios.

Contribution

The paper presents a new augmentation technique for thermal image object detection that effectively integrates synthetic 3D objects with real scenes, improving detection accuracy.

Findings

Achieved state-of-the-art results on the FLIR ADAS dataset.

Demonstrated the effectiveness of combining synthetic and real thermal data.

Compared various augmentation strategies, including RL-based and generative approaches.

Abstract

In this paper we propose a novel data augmentation approach for visual content domains that have scarce training datasets, compositing synthetic 3D objects within real scenes. We show the performance of the proposed system in the context of object detection in thermal videos, a domain where 1) training datasets are very limited compared to visible spectrum datasets and 2) creating full realistic synthetic scenes is extremely cumbersome and expensive due to the difficulty in modeling the thermal properties of the materials of the scene. We compare different augmentation strategies, including state of the art approaches obtained through RL techniques, the injection of simulated data and the employment of a generative model, and study how to best combine our proposed augmentation with these other techniques.Experimental results demonstrate the effectiveness of our approach, and our single-modality detector achieves state-of-the-art results on the FLIR ADAS dataset.