Low-cost Robust Night-time Aerial Material Segmentation through Hyperspectral Data and Sparse Spatio-Temporal Learning

TL;DR

This paper introduces a novel Siamese framework that leverages hyperspectral data and temporal compression to improve night-time aerial material segmentation under challenging lighting and atmospheric conditions.

Contribution

It presents an innovative, scalable approach to integrate low-resolution hyperspectral data into segmentation models using time series compression.

Findings

Achieves competitive results on aerial datasets in various conditions.

Effectively combines spectral and spatial data for improved segmentation.

Demonstrates robustness in poor lighting and atmospheric conditions.

Abstract

Material segmentation is a complex task, particularly when dealing with aerial data in poor lighting and atmospheric conditions. To address this, hyperspectral data from specialized cameras can be very useful in addition to RGB images. However, due to hardware constraints, high spectral data often come with lower spatial resolution. Additionally, incorporating such data into a learning-based segmentation framework is challenging due to the numerous data channels involved. To overcome these difficulties, we propose an innovative Siamese framework that uses time series-based compression to effectively and scalably integrate the additional spectral data into the segmentation task. We demonstrate our model's effectiveness through competitive benchmarks on aerial datasets in various environmental conditions.