Deep intra-operative illumination calibration of hyperspectral cameras

TL;DR

This paper introduces a learning-based method for real-time hyperspectral image recalibration during surgery, addressing lighting variability issues and improving clinical applicability of hyperspectral imaging in surgical settings.

Contribution

It presents a novel automatic recalibration approach that outperforms existing methods, is fast, accurate, and generalizes across different species and lighting conditions.

Findings

Recalibration improves physiological parameter estimation and scene segmentation.

Method outperforms previous approaches in accuracy and speed.

Effective across phantom, porcine, and rat models.

Abstract

Hyperspectral imaging (HSI) is emerging as a promising novel imaging modality with various potential surgical applications. Currently available cameras, however, suffer from poor integration into the clinical workflow because they require the lights to be switched off, or the camera to be manually recalibrated as soon as lighting conditions change. Given this critical bottleneck, the contribution of this paper is threefold: (1) We demonstrate that dynamically changing lighting conditions in the operating room dramatically affect the performance of HSI applications, namely physiological parameter estimation, and surgical scene segmentation. (2) We propose a novel learning-based approach to automatically recalibrating hyperspectral images during surgery and show that it is sufficiently accurate to replace the tedious process of white reference-based recalibration. (3) Based on a total of 742 HSI cubes from a phantom, porcine models, and rats we show that our recalibration method not only outperforms previously proposed methods, but also generalizes across species, lighting conditions, and image processing tasks. Due to its simple workflow integration as well as high accuracy, speed, and generalization capabilities, our method could evolve as a central component in clinical surgical HSI.