MAIS: Memory-Attention for Interactive Segmentation

TL;DR

MAIS introduces a Memory-Attention mechanism that leverages past user inputs and segmentation states to improve the efficiency and accuracy of interactive medical image segmentation, especially when using ViT-based models.

Contribution

The paper proposes a novel Memory-Attention mechanism for interactive segmentation that effectively integrates temporal context from previous interactions, enhancing ViT-based models.

Findings

Improved segmentation accuracy across multiple imaging modalities.

Reduced number of user interactions needed for accurate segmentation.

Enhanced efficiency in interactive medical segmentation tasks.

Abstract

Interactive medical segmentation reduces annotation effort by refining predictions through user feedback. Vision Transformer (ViT)-based models, such as the Segment Anything Model (SAM), achieve state-of-the-art performance using user clicks and prior masks as prompts. However, existing methods treat interactions as independent events, leading to redundant corrections and limited refinement gains. We address this by introducing MAIS, a Memory-Attention mechanism for Interactive Segmentation that stores past user inputs and segmentation states, enabling temporal context integration. Our approach enhances ViT-based segmentation across diverse imaging modalities, achieving more efficient and accurate refinements.