A Classifier-Free Incremental Learning Framework for Scalable Medical Image Segmentation

TL;DR

This paper presents a flexible, classifier-free incremental learning framework for medical image segmentation that adapts to changing classes and data streams, outperforming existing methods in multi-modal, multi-source scenarios.

Contribution

It introduces a novel class-agnostic segmentation network trained with contrastive learning and integrated into a knowledge distillation framework for scalable incremental learning.

Findings

Outperforms state-of-the-art methods on multi-modal datasets

Handles varying class numbers within a single network

Effectively mitigates catastrophic forgetting in incremental learning

Abstract

Current methods for developing foundation models in medical image segmentation rely on two primary assumptions: a fixed set of classes and the immediate availability of a substantial and diverse training dataset. However, this can be impractical due to the evolving nature of imaging technology and patient demographics, as well as labor-intensive data curation, limiting their practical applicability and scalability. To address these challenges, we introduce a novel segmentation paradigm enabling the segmentation of a variable number of classes within a single classifier-free network, featuring an architecture independent of class number. This network is trained using contrastive learning and produces discriminative feature representations that facilitate straightforward interpretation. Additionally, we integrate this strategy into a knowledge distillation-based incremental learning framework, facilitating the gradual assimilation of new information from non-stationary data streams while avoiding catastrophic forgetting. Our approach provides a unified solution for tackling both class- and domain-incremental learning scenarios. We demonstrate the flexibility of our method in handling varying class numbers within a unified network and its capacity for incremental learning. Experimental results on an incompletely annotated, multi-modal, multi-source dataset for medical image segmentation underscore its superiority over state-of-the-art alternative approaches.