Learning Incrementally to Segment Multiple Organs in a CT Image

TL;DR

This paper presents an incremental learning approach for multi-organ segmentation in CT images, effectively updating models with new data over time without catastrophic forgetting, and introduces techniques to stabilize performance across stages.

Contribution

The paper introduces a novel incremental learning method for multi-organ CT segmentation that mitigates catastrophic forgetting and stabilizes model performance using a light memory module and specialized loss functions.

Findings

Catastrophic forgetting is less severe in CT multi-organ segmentation.

The proposed method outperforms existing approaches on five datasets.

Stabilization techniques improve model consistency across incremental stages.

Abstract

There exists a large number of datasets for organ segmentation, which are partially annotated and sequentially constructed. A typical dataset is constructed at a certain time by curating medical images and annotating the organs of interest. In other words, new datasets with annotations of new organ categories are built over time. To unleash the potential behind these partially labeled, sequentially-constructed datasets, we propose to incrementally learn a multi-organ segmentation model. In each incremental learning (IL) stage, we lose the access to previous data and annotations, whose knowledge is assumingly captured by the current model, and gain the access to a new dataset with annotations of new organ categories, from which we learn to update the organ segmentation model to include the new organs. While IL is notorious for its `catastrophic forgetting' weakness in the context of natural image analysis, we experimentally discover that such a weakness mostly disappears for CT multi-organ segmentation. To further stabilize the model performance across the IL stages, we introduce a light memory module and some loss functions to restrain the representation of different categories in feature space, aggregating feature representation of the same class and separating feature representation of different classes. Extensive experiments on five open-sourced datasets are conducted to illustrate the effectiveness of our method.