MediRound: Multi-Round Entity-Level Reasoning Segmentation in Medical Images

TL;DR

MediRound introduces a new multi-round reasoning segmentation task for medical images, supported by a large dataset and a novel model with a correction mechanism, advancing multi-turn medical image understanding.

Contribution

The paper defines the MEMR-Seg task, creates the MR-MedSeg dataset, and proposes MediRound with a judgment and correction mechanism for improved multi-round segmentation.

Findings

MediRound outperforms existing segmentation methods on MEMR-Seg.

The correction mechanism reduces error propagation in multi-round reasoning.

The dataset enables research in multi-turn medical image understanding.

Abstract

Despite recent progress in text-prompt-based medical image segmentation, these methods are limited to single-round dialogues and fail to support multi-round reasoning, which is important for medical education scenarios. In this work, we introduce Multi-Round Entity-Level Medical Reasoning Segmentation (MEMR-Seg), a new task that requires generating segmentation masks through multi-round queries with entity-level reasoning, helping learners progressively develop their understanding of medical knowledge. To support this task, we construct MR-MedSeg, a large-scale dataset of 177K multi-round medical segmentation dialogues, featuring entity-based reasoning across rounds. Furthermore, we propose MediRound, an effective baseline model designed for multi-round medical reasoning segmentation. To mitigate the inherent error propagation within the chain-like pipeline of multi-round segmentation, we introduce a lightweight yet effective Judgment & Correction Mechanism during model inference. Experimental results demonstrate that our method effectively addresses the MEMR-Seg task and outperforms conventional medical referring segmentation methods. The project is available at https://github.com/Edisonhimself/MediRound.