Thermal Image Processing via Physics-Inspired Deep Networks

TL;DR

DeepIR is a physics-inspired deep learning framework for thermal image processing that effectively performs denoising and super-resolution without requiring training data or calibration, leveraging physical sensor models and deep regularizers.

Contribution

The paper introduces DeepIR, a novel thermal image processing method combining physics-based sensor modeling with deep networks, eliminating the need for training data or calibration.

Findings

Achieves high-quality non-uniformity correction with as few as three images.

Improves PSNR by 10dB over existing methods in experiments.

Effectively performs denoising and super-resolution on thermal images.

Abstract

We introduce DeepIR, a new thermal image processing framework that combines physically accurate sensor modeling with deep network-based image representation. Our key enabling observations are that the images captured by thermal sensors can be factored into slowly changing, scene-independent sensor non-uniformities (that can be accurately modeled using physics) and a scene-specific radiance flux (that is well-represented using a deep network-based regularizer). DeepIR requires neither training data nor periodic ground-truth calibration with a known black body target--making it well suited for practical computer vision tasks. We demonstrate the power of going DeepIR by developing new denoising and super-resolution algorithms that exploit multiple images of the scene captured with camera jitter. Simulated and real data experiments demonstrate that DeepIR can perform high-quality non-uniformity correction with as few as three images, achieving a 10dB PSNR improvement over competing approaches.