Hyper-Drive: Visible-Short Wave Infrared Hyperspectral Imaging Datasets for Robots in Unstructured Environments

TL;DR

This paper introduces a novel hyperspectral imaging system for robots, providing a large, labeled dataset across visible to shortwave infrared spectra to enhance terrain understanding in unstructured environments.

Contribution

The work presents a new system architecture with hyperspectral cameras for robots and releases a comprehensive dataset to advance hyperspectral applications in robotics.

Findings

Hyperspectral data improves terrain classification accuracy.

The dataset enables better scene understanding from a robot perspective.

Open-source code and data facilitate further research.

Abstract

Hyperspectral sensors have enjoyed widespread use in the realm of remote sensing; however, they must be adapted to a format in which they can be operated onboard mobile robots. In this work, we introduce a first-of-its-kind system architecture with snapshot hyperspectral cameras and point spectrometers to efficiently generate composite datacubes from a robotic base. Our system collects and registers datacubes spanning the visible to shortwave infrared (660-1700 nm) spectrum while simultaneously capturing the ambient solar spectrum reflected off a white reference tile. We collect and disseminate a large dataset of more than 500 labeled datacubes from on-road and off-road terrain compliant with the ATLAS ontology to further the integration and demonstration of hyperspectral imaging (HSI) as beneficial in terrain class separability. Our analysis of this data demonstrates that HSI is a significant opportunity to increase understanding of scene composition from a robot-centric context. All code and data are open source online: https://river-lab.github.io/hyper_drive_data