Deep Learning-Based Correction and Unmixing of Hyperspectral Images for Brain Tumor Surgery

TL;DR

This paper introduces two deep learning models that improve hyperspectral image correction and unmixing in brain tumor surgery, leading to more accurate fluorophore quantification and better tissue visualization.

Contribution

The study develops novel autoencoder-based deep learning models for nonlinear correction and unmixing of hyperspectral images, outperforming classical linear methods in accuracy and realism.

Findings

Supervised model achieved Pearson R of 0.997 and 0.990 on phantom and pig data.

Semi-supervised model achieved Pearson R of 0.98 and 0.91, respectively.

Models produced more realistic and consistent PpIX abundance estimates on human data.

Abstract

Hyperspectral Imaging (HSI) for fluorescence-guided brain tumor resection enables visualization of differences between tissues that are not distinguishable to humans. This augmentation can maximize brain tumor resection, improving patient outcomes. However, much of the processing in HSI uses simplified linear methods that are unable to capture the non-linear, wavelength-dependent phenomena that must be modeled for accurate recovery of fluorophore abundances. We therefore propose two deep learning models for correction and unmixing, which can account for the nonlinear effects and produce more accurate estimates of abundances. Both models use an autoencoder-like architecture to process the captured spectra. One is trained with protoporphyrin IX (PpIX) concentration labels. The other undergoes semi-supervised training, first learning hyperspectral unmixing self-supervised and then learning to correct fluorescence emission spectra for heterogeneous optical and geometric properties using a reference white-light reflectance spectrum in a few-shot manner. The models were evaluated against phantom and pig brain data with known PpIX concentration; the supervised model achieved Pearson correlation coefficients (R values) between the known and computed PpIX concentrations of 0.997 and 0.990, respectively, whereas the classical approach achieved only 0.93 and 0.82. The semi-supervised approach's R values were 0.98 and 0.91, respectively. On human data, the semi-supervised model gives qualitatively more realistic results than the classical method, better removing bright spots of specular reflectance and reducing the variance in PpIX abundance over biopsies that should be relatively homogeneous. These results show promise for using deep learning to improve HSI in fluorescence-guided neurosurgery.