On RGB-TIR Stereo Calibration under Extreme Resolution Asymmetry

TL;DR

This paper introduces a practical stereo calibration method for RGB-TIR systems with extreme resolution asymmetry, using an active display and advanced detection algorithms for accurate geometric calibration.

Contribution

It presents a novel calibration framework that handles ultra-low resolution thermal images and ensures physically consistent stereo geometry for building analysis.

Findings

Achieved a baseline of 32.7 mm with 0.382 px reprojection error.

Reliable checkerboard detection at 80 x 62 px without per-frame tuning.

Validated system on a building mock-up for energy performance assessment.

Abstract

Accurate geometric calibration of RGB-thermal infrared (TIR) stereo camera systems is essential for multimodal building envelope analysis, yet remains challenging when low-cost thermal sensors with very low spatial resolution are employed. This paper presents a practical stereo calibration framework for an RGB camera (2028 x 1520 px) paired with a TIR camera operating at only 80 x 62 px - a pixel-count ratio of approximately 1:625. An active OLED screen dynamically switches modality-specific patterns (checkerboard for TIR, ChArUco for RGB) on a single physical surface, providing controlled and repeatable thermal contrast. A dedicated corner detection algorithm combining perspective rectification, Hessian saddle-point analysis, and Mean Shift localisation achieves reliable checkerboard detection at 80 x 62 px without per-frame parameter tuning. A baseline-constrained bundle adjustment enforces physically consistent rig geometry under the planar-calibration-object degeneracy, yielding a stereo baseline of 32.7 mm (nominal 30 mm) with an overall reprojection error of 0.382 px. The system is validated on a thermally active building mock-up using constant-depth and per-pixel depth estimation, demonstrating consistent TIR-to-RGB projection suitable for building energy performance assessment.