VerSe: Integrating Multiple Queries as Prompts for Versatile Cardiac MRI Segmentation

TL;DR

VerSe introduces a unified framework that combines automatic and interactive prompts for versatile cardiac MRI segmentation, significantly improving accuracy and efficiency over existing methods.

Contribution

It proposes a novel joint learning approach for object and click queries as prompts, enabling seamless integration of automatic and interactive segmentation in cardiac MRI.

Findings

Outperforms existing methods in accuracy and efficiency

Effective on both cardiac MRI and diverse medical datasets

Supports both automatic and interactive segmentation modes

Abstract

Despite the advances in learning-based image segmentation approach, the accurate segmentation of cardiac structures from magnetic resonance imaging (MRI) remains a critical challenge. While existing automatic segmentation methods have shown promise, they still require extensive manual corrections of the segmentation results by human experts, particularly in complex regions such as the basal and apical parts of the heart. Recent efforts have been made on developing interactive image segmentation methods that enable human-in-the-loop learning. However, they are semi-automatic and inefficient, due to their reliance on click-based prompts, especially for 3D cardiac MRI volumes. To address these limitations, we propose VerSe, a Versatile Segmentation framework to unify automatic and interactive segmentation through mutiple queries. Our key innovation lies in the joint learning of object and click queries as prompts for a shared segmentation backbone. VerSe supports both fully automatic segmentation, through object queries, and interactive mask refinement, by providing click queries when needed. With the proposed integrated prompting scheme, VerSe demonstrates significant improvement in performance and efficiency over existing methods, on both cardiac MRI and out-of-distribution medical imaging datasets. The code is available at https://github.com/bangwayne/Verse.