RAIS: Robust and Accurate Interactive Segmentation via Continual Learning

TL;DR

RAIS introduces a continual learning approach for interactive image segmentation, enabling models to adapt to domain shifts during inference, thereby improving accuracy and robustness across diverse datasets.

Contribution

The paper presents a novel architecture with an optimization strategy for continual learning, enhancing adaptability and performance in interactive segmentation tasks.

Findings

Achieves state-of-the-art performance on multiple benchmarks.

Handles data distribution shifts effectively.

Demonstrates robustness in remote sensing and medical imaging datasets.

Abstract

Interactive image segmentation aims at segmenting a target region through a way of human-computer interaction. Recent works based on deep learning have achieved excellent performance, while most of them focus on improving the accuracy of the training set and ignore potential improvement on the test set. In the inference phase, they tend to have a good performance on similar domains to the training set, and lack adaptability to domain shift, so they require more user efforts to obtain satisfactory results. In this work, we propose RAIS, a robust and accurate architecture for interactive segmentation with continuous learning, where the model can learn from both train and test data sets. For efficient learning on the test set, we propose a novel optimization strategy to update global and local parameters with a basic segmentation module and adaptation module, respectively. Moreover, we perform extensive experiments on several benchmarks that show our method can handle data distribution shifts and achieves SOTA performance compared with recent interactive segmentation methods. Besides, our method also shows its robustness in the datasets of remote sensing and medical imaging where the data domains are completely different between training and testing.