Hyperspectral Reconstruction of Skin Through Fusion of Scattering Transform Features

TL;DR

This paper introduces a novel hyperspectral skin reconstruction method using scattering transform features, enhancing the learning process by matching and inverting features instead of pixel values, aimed at overcoming spectral device limitations.

Contribution

The paper presents a new model leveraging scattering transform features for hyperspectral skin reconstruction from RGB and infrared images, improving efficiency and accuracy.

Findings

Effective reconstruction of skin hyperspectral images from RGB and infrared data.

Reduced complexity in feature matching improves learning performance.

Potential for practical applications in medical imaging and skin analysis.

Abstract

Hyperspectral imagery (HSI) is an established technique with an array of applications, but its use is limited due to both practical and technical issues associated with spectral devices. The goal of the ICASSP 2024 'Hyper-Skin' Challenge is to extract skin HSI from matching RGB images and an infrared band. To address this problem we propose a model using features of the scattering transform - a type of convolutional neural network with predefined filters. Our model matches and inverts those features, rather than the pixel values, reducing the complexity of matching while grouping similar features together, resulting in an improved learning process.