Domain Adaptive Sim-to-Real Segmentation of Oropharyngeal Organs

TL;DR

This paper introduces a domain adaptive framework combining image blending and style transfer to improve segmentation of oropharyngeal organs in endoscopic images, addressing data scarcity and domain differences.

Contribution

The paper proposes IRB-AF, a novel domain adaptive Sim-to-Real framework that enhances segmentation accuracy using IoU-Ranking Blend and ArtFlow style transfer techniques.

Findings

Improved segmentation accuracy over baseline models.

Enhanced training stability in domain adaptation.

Effective handling of dataset domain discrepancies.

Abstract

Video-assisted transoral tracheal intubation (TI) necessitates using an endoscope that helps the physician insert a tracheal tube into the glottis instead of the esophagus. The growing trend of robotic-assisted TI would require a medical robot to distinguish anatomical features like an experienced physician which can be imitated by utilizing supervised deep-learning techniques. However, the real datasets of oropharyngeal organs are often inaccessible due to limited open-source data and patient privacy. In this work, we propose a domain adaptive Sim-to-Real framework called IoU-Ranking Blend-ArtFlow (IRB-AF) for image segmentation of oropharyngeal organs. The framework includes an image blending strategy called IoU-Ranking Blend (IRB) and style-transfer method ArtFlow. Here, IRB alleviates the problem of poor segmentation performance caused by significant datasets domain differences; while ArtFlow is introduced to reduce the discrepancies between datasets further. A virtual oropharynx image dataset generated by the SOFA framework is used as the learning subject for semantic segmentation to deal with the limited availability of actual endoscopic images. We adapted IRB-AF with the state-of-the-art domain adaptive segmentation models. The results demonstrate the superior performance of our approach in further improving the segmentation accuracy and training stability.