Exploring Fully Convolutional Networks for the Segmentation of Hyperspectral Imaging Applied to Advanced Driver Assistance Systems

TL;DR

This paper investigates the use of fully convolutional networks for segmenting hyperspectral images in advanced driver assistance systems, aiming to improve object separation under challenging conditions using spectral and spatial features.

Contribution

It introduces a novel application of FCNs to hyperspectral image segmentation for ADAS, demonstrating potential improvements in challenging driving scenarios.

Findings

FCNs effectively utilize spectral and spatial features for HSI segmentation.

Experimental results show promising accuracy in real driving conditions.

Prototype implementation indicates feasible real-time deployment on MPSoC.

Abstract

Advanced Driver Assistance Systems (ADAS) are designed with the main purpose of increasing the safety and comfort of vehicle occupants. Most of current computer vision-based ADAS perform detection and tracking tasks quite successfully under regular conditions, but are not completely reliable, particularly under adverse weather and changing lighting conditions, neither in complex situations with many overlapping objects. In this work we explore the use of hyperspectral imaging (HSI) in ADAS on the assumption that the distinct near infrared (NIR) spectral reflectances of different materials can help to better separate the objects in a driving scene. In particular, this paper describes some experimental results of the application of fully convolutional networks (FCN) to the image segmentation of HSI for ADAS applications. More specifically, our aim is to investigate to what extent the spatial features codified by convolutional filters can be helpful to improve the performance of HSI segmentation systems. With that aim, we use the HSI-Drive v1.1 dataset, which provides a set of labelled images recorded in real driving conditions with a small-size snapshot NIR-HSI camera. Finally, we analyze the implementability of such a HSI segmentation system by prototyping the developed FCN model together with the necessary hyperspectral cube preprocessing stage and characterizing its performance on an MPSoC.