Beyond Manual Annotation: A Human-AI Collaborative Framework for Medical Image Segmentation Using Only "Better or Worse" Expert Feedback

TL;DR

This paper presents a human-AI collaborative framework for medical image segmentation that reduces manual labeling by using only binary better-or-worse feedback from experts, enabling efficient training of segmentation models.

Contribution

It introduces a preference learning paradigm and a multi-component framework that leverages minimal expert feedback to train segmentation models without explicit manual annotations.

Findings

Achieves competitive segmentation performance with minimal feedback.

Eliminates the need for manual pixel-level annotations.

Demonstrates effectiveness on multiple public datasets.

Abstract

Manual annotation of medical images is a labor-intensive and time-consuming process, posing a significant bottleneck in the development and deployment of robust medical imaging AI systems. This paper introduces a novel hands-free Human-AI collaborative framework for medical image segmentation that substantially reduces the annotation burden by eliminating the need for explicit manual pixel-level labeling. The core innovation lies in a preference learning paradigm, where human experts provide minimal, intuitive feedback -- simply indicating whether an AI-generated segmentation is better or worse than a previous version. The framework comprises four key components: (1) an adaptable foundation model (FM) for feature extraction, (2) label propagation based on feature similarity, (3) a clicking agent that learns from human better-or-worse feedback to decide where to click and with which label, and (4) a multi-round segmentation learning procedure that trains a state-of-the-art segmentation network using pseudo-labels generated by the clicking agent and FM-based label propagation. Experiments on three public datasets demonstrate that the proposed approach achieves competitive segmentation performance using only binary preference feedback, without requiring experts to directly manually annotate the images.