Super-resolution for localizing electrode grids as small, deformable objects during epilepsy surgery using augmented reality headsets

TL;DR

This paper introduces a new method using AR headsets and AI to accurately locate small, deformable electrode grids during epilepsy surgery.

Contribution

A novel method combining super-resolution, AI, and AR for localizing deformable objects in surgical settings.

Findings

The system achieved sub-5 mm accuracy in localizing the ioECoG grid at distances under 60 cm.

At 40 cm, accuracy remained below 2 mm with a standard deviation under 0.5 mm.

The method processed 24 stereo frames per second on average at 60 cm.

Abstract

Epilepsy surgery is a potential curative treatment for people with focal epilepsy. Intraoperative electrocorticogram (ioECoG) recordings from the brain guide neurosurgeons during resection. Accurate localization of epileptic activity and thus the ioECoG grids is critical for successful outcomes. We aim to develop and evaluate the feasibility of a novel method for localizing small, deformable objects using augmented reality (AR) head-mounted displays (HMDs) and artificial intelligence (AI). AR HMDs combine cameras and patient overlay visualization in a compact design.

We developed an image processing method for the HoloLens 2 to localize a 64-electrode ioECoG grid even when individual electrodes are indistinguishable due to low resolution. The method combines object detection, super-resolution, and pose estimation AI models with stereo triangulation. A synthetic dataset of 90,000 images trained the super-resolution and pose estimation models. The system was tested in a controlled environment against an optical tracker as ground truth. Accuracy was evaluated at distances between 40 and 90 cm.

The system achieved sub-5 mm accuracy in localizing the ioECoG grid at distances shorter than 60 cm. At 40 cm, the accuracy remained below 2 mm, with an average standard deviation of less than 0.5 mm. At 60 cm the method processed on average 24 stereo frames per second.

This study demonstrates the feasibility of localizing small, deformable objects like ioECoG grids using AR HMDs. While results indicate clinically acceptable accuracy, further research is needed to validate the method in clinical environments and assess its impact on surgical precision and outcomes.

Figures