Human Gaze-based Dual Teacher Guidance Learning for Semi-Supervised Medical Image Segmentation

TL;DR

This paper introduces HG-DTGL, a semi-supervised medical image segmentation method that leverages human gaze data as an additional teacher to improve dataset diversity and network perception.

Contribution

It proposes a novel gaze-based dual teacher framework incorporating GazeMix, MGP, and Gaze Loss to enhance segmentation accuracy with limited labeled data.

Findings

Achieved superior segmentation performance across multiple datasets and organs.

Demonstrated strong generalization across different medical imaging modalities.

Validated the effectiveness of gaze data in semi-supervised learning.

Abstract

In the field of medical image segmentation, the scarcity of labeled data poses a major challenge for existing models to accurately perceive target regions. Compared with manual annotation, gaze data is easier and cheaper to obtain. As a classical semi-supervised learning framework, mean-teacher can effectively use a large number of unlabeled medical images for stable training through self-teaching and collaborative optimization. Our study is based on the mean-teacher framework. By combining gaze data, it aims to address two crucial issues in semi-supervised medical image segmentation: 1) expand the scale and diversity of the dataset with limited labeled data; 2) enhance the network's perception ability. We propose the Human Gaze-based Dual Teacher Guidance Learning model (HG-DTGL). In this model, human gaze serves as an additional hidden `teacher' in the mean-teacher architecture. We introduce the GazeMix to generate reliable mixed data to expand the diversity and scale of the dataset, and the Multi-scale Gaze Perception (MGP) module is used to extract the multi-scale perception of the network. A Gaze Loss is designed to align the model's perception with human gaze. We have verified HG-DTGL on multiple datasets of different modalities and achieved superior performance on a total of ten different organs/tissues, with extensive experiments. This demonstrates that our method has strong generalization ability for medical images of different modalities, and shows the great application potential of gaze data in semi-supervised medical image segmentation.