# The AENEAS Project: Intraoperative Anatomical Guidance Through Real-Time Landmark Detection Using Machine Vision

## TL;DR

This paper explores using machine vision to detect anatomical landmarks during complex brain surgery, showing promising results for deep structures like the optic nerve.

## Contribution

The study introduces a deep learning model for real-time anatomical landmark detection in microsurgery, validated with a large dataset of surgical videos.

## Key findings

- The model achieved high precision for detecting deep structures like the optic nerve (AP50: 0.73) and internal carotid artery (AP50: 0.67).
- Superficial structures had lower precision due to morphological similarity and optical variability.
- Performance variability highlights the complexity of the anatomical setting and data limitations.

## Abstract

To investigate the performance of a deep learning machine vision-based model in identifying anatomical landmarks in a complex microsurgical setting, such as the pterional trans-Sylvian approach.

We developed a deep learning object detection model (YOLOv7x) trained on 5307 labeled frames from 78 surgical videos of 76 patients undergoing pterional trans-Sylvian approach from January 1, 2020 to June 31, 2024. Surgical steps were standardized, and key anatomical targets—frontal/temporal dura, inferior frontal/superior temporal gyri, optic and olfactory nerves, and internal carotid artery—were annotated by specifically trained neurosurgical residents and verified by the operating surgeon. Bounding boxes derived from segmentation masks served as training inputs. Performance was evaluated using 5-fold cross-validation.

The model achieved promising detection performance for deep structures, particularly the optic nerve (average precision at an intersection over union threshold of 0.50 [AP50]: 0.73) and internal carotid artery (AP50: 0.67). Superficial structures, like dura and cortical gyri, had lower precision (AP50 range: 0.25-0.45), likely due to morphological similarity and optical variability. Performance variability across classes reflects the complexity of the anatomical setting along with data limitations.

Applying machine vision techniques for anatomical detection in a complex neurosurgical setting is feasible. Although challenges remain in detecting less distinctive structures, the high accuracy achieved for deep anatomical landmarks validates this approach. This study marks an essential step toward the development of machine vision-powered anatomical recognition tools, with the prospective goal of improving intraoperative orientation and reducing variability among surgeons.