F2PASeg: Feature Fusion for Pituitary Anatomy Segmentation in Endoscopic Surgery

TL;DR

F2PASeg introduces a feature fusion-based deep learning model and a new dataset to improve real-time segmentation of pituitary anatomy during endoscopic surgery, aiding surgical safety.

Contribution

The paper presents a novel dataset for pituitary anatomy segmentation and a feature fusion module that enhances segmentation robustness during surgery.

Findings

F2PASeg achieves accurate real-time segmentation of critical structures.

Data augmentation improves model performance under occlusions and bleeding.

The method demonstrates robustness against intraoperative variations.

Abstract

Pituitary tumors often cause deformation or encapsulation of adjacent vital structures. Anatomical structure segmentation can provide surgeons with early warnings of regions that pose surgical risks, thereby enhancing the safety of pituitary surgery. However, pixel-level annotated video stream datasets for pituitary surgeries are extremely rare. To address this challenge, we introduce a new dataset for Pituitary Anatomy Segmentation (PAS). PAS comprises 7,845 time-coherent images extracted from 120 videos. To mitigate class imbalance, we apply data augmentation techniques that simulate the presence of surgical instruments in the training data. One major challenge in pituitary anatomy segmentation is the inconsistency in feature representation due to occlusions, camera motion, and surgical bleeding. By incorporating a Feature Fusion module, F2PASeg is proposed to refine anatomical structure segmentation by leveraging both high-resolution image features and deep semantic embeddings, enhancing robustness against intraoperative variations. Experimental results demonstrate that F2PASeg consistently segments critical anatomical structures in real time, providing a reliable solution for intraoperative pituitary surgery planning. Code: https://github.com/paulili08/F2PASeg.