Automatic Tissue Differentiation in Parotidectomy using Hyperspectral Imaging

TL;DR

This study demonstrates that hyperspectral imaging combined with neural networks can accurately differentiate tissues during parotidectomy, potentially aiding surgeons in avoiding injury to critical structures.

Contribution

The paper introduces a novel application of hyperspectral imaging with a 3D CNN for intraoperative tissue differentiation in head and neck surgery.

Findings

Achieved 98.7% accuracy in training data

Achieved 83.4% accuracy in unseen patient data

High sensitivity in identifying glandular tissue and nerves

Abstract

In head and neck surgery, continuous intraoperative tissue differentiation is of great importance to avoid injury to sensitive structures such as nerves and vessels. Hyperspectral imaging (HSI) with neural network analysis could support the surgeon in tissue differentiation. A 3D Convolutional Neural Network with hyperspectral data in the range of $400-1000$ nm is used in this work. The acquisition system consisted of two multispectral snapshot cameras creating a stereo-HSI-system. For the analysis, 27 images with annotations of glandular tissue, nerve, muscle, skin and vein in 18 patients undergoing parotidectomy are included. Three patients are removed for evaluation following the leave-one-subject-out principle. The remaining images are used for training, with the data randomly divided into a training group and a validation group. In the validation, an overall accuracy of $98.7\%$ is achieved, indicating robust training. In the evaluation on the excluded patients, an overall accuracy of $83.4\%$ has been achieved showing good detection and identification abilities. The results clearly show that it is possible to achieve robust intraoperative tissue differentiation using hyperspectral imaging. Especially the high sensitivity in parotid or nerve tissue is of clinical importance. It is interesting to note that vein was often confused with muscle. This requires further analysis and shows that a very good and comprehensive data basis is essential. This is a major challenge, especially in surgery.