Dual-Alignment Knowledge Retention for Continual Medical Image Segmentation

TL;DR

This paper proposes a dual-alignment framework for continual medical image segmentation that reduces catastrophic forgetting by aligning features across networks and data representations, improving performance across domains.

Contribution

It introduces a novel dual-alignment strategy with CNA and CRA modules, leveraging HSIC analysis to better capture complex task dependencies in continual learning.

Findings

Effective in mitigating catastrophic forgetting under domain shifts

Outperforms existing methods on medical image segmentation tasks

Demonstrates robustness across diverse clinical domains

Abstract

Continual learning in medical image segmentation involves sequential data acquisition across diverse domains (e.g., clinical sites), where task interference between past and current domains often leads to catastrophic forgetting. Existing continual learning methods fail to capture the complex dependencies between tasks. We introduce a novel framework that mitigates forgetting by establishing and enhancing complex dependencies between historical data and the network in the present task. Our framework features a dual-alignment strategy, the cross-network alignment (CNA) module aligns the features extracted from the bottleneck layers of the current and previous networks, respectively, while the cross-representation alignment (CRA) module aligns the features learned by the current network from historical buffered data and current input data, respectively. Implementing both types of alignment is a non-trivial task. To address this, we further analyze the linear and nonlinear forms of the well-established Hilbert-Schmidt Independence Criterion (HSIC) and deliberately design feature mapping and feature pairing blocks within the CRA module. Experiments on medical image segmentation task demonstrate our framework's effectiveness in mitigating catastrophic forgetting under domain shifts.