Deep Fusion of Ultra-Low-Resolution Thermal Camera and Gyroscope Data for Lighting-Robust and Compute-Efficient Rotational Odometry

TL;DR

This paper presents a novel sensor fusion method combining ultra-low-resolution thermal imaging with gyroscope data to achieve lighting-robust, efficient rotational odometry suitable for resource-constrained robots.

Contribution

The study introduces thermal-gyro fusion, a new approach that reduces thermal camera resolution while maintaining accuracy, enhancing efficiency for real-time robotic applications.

Findings

Thermal-gyro fusion significantly reduces thermal camera resolution requirements.

The proposed CNN effectively fuses modalities for accurate rotational speed estimation.

Resource efficiency is improved without major accuracy loss.

Abstract

Accurate rotational odometry is crucial for autonomous robotic systems, particularly for small, power-constrained platforms such as drones and mobile robots. This study introduces thermal-gyro fusion, a novel sensor fusion approach that integrates ultra-low-resolution thermal imaging with gyroscope readings for rotational odometry. Unlike RGB cameras, thermal imaging is invariant to lighting conditions and, when fused with gyroscopic data, mitigates drift which is a common limitation of inertial sensors. We first develop a multimodal data acquisition system to collect synchronized thermal and gyroscope data, along with rotational speed labels, across diverse environments. Subsequently, we design and train a lightweight Convolutional Neural Network (CNN) that fuses both modalities for rotational speed estimation. Our analysis demonstrates that thermal-gyro fusion enables a significant reduction in thermal camera resolution without significantly compromising accuracy, thereby improving computational efficiency and memory utilization. These advantages make our approach well-suited for real-time deployment in resource-constrained robotic systems. Finally, to facilitate further research, we publicly release our dataset as supplementary material.