Deep Reinforcement Learning Based System for Intraoperative Hyperspectral Video Autofocusing

TL;DR

This paper introduces a deep reinforcement learning-based autofocus system for hyperspectral intraoperative video, improving focus accuracy and usability in surgical settings by integrating a focus-tunable lens and benchmarking against traditional methods.

Contribution

It presents a novel deep reinforcement learning autofocus algorithm for hyperspectral video, validated with a robotic focal scan dataset and usability trials with neurosurgeons.

Findings

The proposed algorithm outperforms traditional autofocus policies in mean absolute focal error.

The system received favorable feedback from neurosurgeons in usability trials.

Benchmarking shows significant improvement ($p<0.05$) over traditional methods.

Abstract

Hyperspectral imaging (HSI) captures a greater level of spectral detail than traditional optical imaging, making it a potentially valuable intraoperative tool when precise tissue differentiation is essential. Hardware limitations of current optical systems used for handheld real-time video HSI result in a limited focal depth, thereby posing usability issues for integration of the technology into the operating room. This work integrates a focus-tunable liquid lens into a video HSI exoscope, and proposes novel video autofocusing methods based on deep reinforcement learning. A first-of-its-kind robotic focal-time scan was performed to create a realistic and reproducible testing dataset. We benchmarked our proposed autofocus algorithm against traditional policies, and found our novel approach to perform significantly ($p<0.05$) better than traditional techniques ($0.070\pm.098$ mean absolute focal error compared to $0.146\pm.148$). In addition, we performed a blinded usability trial by having two neurosurgeons compare the system with different autofocus policies, and found our novel approach to be the most favourable, making our system a desirable addition for intraoperative HSI.