Online Reflective Learning for Robust Medical Image Segmentation

TL;DR

This paper introduces RefSeg, an online reflective learning framework that enhances the robustness of medical image segmentation models by enabling them to reflect on their failures during testing, inspired by human learning cycles.

Contribution

RefSeg is a novel online learning framework that synthesizes proxy images to help models recognize and reflect on segmentation failures, improving robustness without retraining.

Findings

RefSeg significantly improves segmentation robustness across multiple medical imaging datasets.

The framework achieves state-of-the-art performance compared to existing methods.

RefSeg is general and applicable to various segmentation models.

Abstract

Deep segmentation models often face the failure risks when the testing image presents unseen distributions. Improving model robustness against these risks is crucial for the large-scale clinical application of deep models. In this study, inspired by human learning cycle, we propose a novel online reflective learning framework (RefSeg) to improve segmentation robustness. Based on the reflection-on-action conception, our RefSeg firstly drives the deep model to take action to obtain semantic segmentation. Then, RefSeg triggers the model to reflect itself. Because making deep models realize their segmentation failures during testing is challenging, RefSeg synthesizes a realistic proxy image from the semantic mask to help deep models build intuitive and effective reflections. This proxy translates and emphasizes the segmentation flaws. By maximizing the structural similarity between the raw input and the proxy, the reflection-on-action loop is closed with segmentation robustness improved. RefSeg runs in the testing phase and is general for segmentation models. Extensive validation on three medical image segmentation tasks with a public cardiac MR dataset and two in-house large ultrasound datasets show that our RefSeg remarkably improves model robustness and reports state-of-the-art performance over strong competitors.