Sub-pixel matching method for low-resolution thermal stereo images

TL;DR

This paper introduces a sub-pixel stereo matching method for low-resolution thermal images, significantly improving feature extraction and matching accuracy for localization and tracking applications.

Contribution

The paper presents a novel threefold sub-pixel matching framework using phase congruency, enhancing feature detection and matching precision in low-res thermal stereo images.

Findings

Extracted twice as many features as state-of-the-art methods.

Achieved four times more matches than baseline methods.

Reduced positional error in 3D localization from over 1000 mm to 300 mm.

Abstract

In the context of a localization and tracking application, we developed a stereo vision system based on cheap low-resolution 80x60 pixels thermal cameras. We proposed a threefold sub-pixel stereo matching framework (called ST for Subpixel Thermal): 1) robust features extraction method based on phase congruency, 2) rough matching of these features in pixel precision, and 3) refined matching in sub-pixel accuracy based on local phase coherence. We performed experiments on our very low-resolution thermal images (acquired using a stereo system we manufactured) as for high-resolution images from a benchmark dataset. Even if phase congruency computation time is high, it was able to extract two times more features than state-of-the-art methods such as ORB or SURF. We proposed a modified version of the phase correlation applied in the phase congruency feature space for sub-pixel matching. Using simulated stereo, we investigated how the phase congruency threshold and the sub-image size of sub-pixel matching can influence the accuracy. We then proved that given our stereo setup and the resolution of our images, being wrong of 1 pixel leads to a 500 mm error in the Z position of the point. Finally, we showed that our method could extract four times more matches than a baseline method ORB + OpenCV KNN matching on low-resolution images. Moreover, our matches were more robust. More precisely, when projecting points of a standing person, ST got a standard deviation of 300 mm when ORB + OpenCV KNN gave more than 1000 mm.