A Multi-spectral Dataset for Evaluating Motion Estimation Systems

TL;DR

This paper introduces a comprehensive multi-spectral dataset combining visible, thermal, depth, and inertial data for evaluating motion estimation systems across diverse lighting conditions.

Contribution

It provides a novel, publicly available dataset with synchronized multi-spectral and depth data, including challenging illumination scenarios, for advancing motion estimation research.

Findings

Dataset includes synchronized multi-spectral, depth, and IMU data.

Provides ground-truth camera poses for accurate evaluation.

Includes diverse lighting conditions for robust testing.

Abstract

Visible images have been widely used for motion estimation. Thermal images, in contrast, are more challenging to be used in motion estimation since they typically have lower resolution, less texture, and more noise. In this paper, a novel dataset for evaluating the performance of multi-spectral motion estimation systems is presented. All the sequences are recorded from a handheld multi-spectral device. It consists of a standard visible-light camera, a long-wave infrared camera, an RGB-D camera, and an inertial measurement unit (IMU). The multi-spectral images, including both color and thermal images in full sensor resolution (640 x 480), are obtained from a standard and a long-wave infrared camera at 32Hz with hardware-synchronization. The depth images are captured by a Microsoft Kinect2 and can have benefits for learning cross-modalities stereo matching. For trajectory evaluation, accurate ground-truth camera poses obtained from a motion capture system are provided. In addition to the sequences with bright illumination, the dataset also contains dim, varying, and complex illumination scenes. The full dataset, including raw data and calibration data with detailed data format specifications, is publicly available.