Multi-scale Feature Alignment for Continual Learning of Unlabeled Domains

TL;DR

This paper introduces a continual learning approach for unsupervised domain adaptation in medical imaging, using generative feature replay and a dual-purpose discriminator to sequentially adapt to multiple unlabeled histopathological domains without storing previous data.

Contribution

It proposes a novel method combining generative feature-driven image replay with a dual-purpose discriminator for continual UDA in medical imaging, addressing data privacy and sequential domain adaptation challenges.

Findings

Achieved state-of-the-art results on histopathological datasets.

Demonstrated effective sequential adaptation without storing previous data.

Validated approach on tissue-type classification and segmentation tasks.

Abstract

Methods for unsupervised domain adaptation (UDA) help to improve the performance of deep neural networks on unseen domains without any labeled data. Especially in medical disciplines such as histopathology, this is crucial since large datasets with detailed annotations are scarce. While the majority of existing UDA methods focus on the adaptation from a labeled source to a single unlabeled target domain, many real-world applications with a long life cycle involve more than one target domain. Thus, the ability to sequentially adapt to multiple target domains becomes essential. In settings where the data from previously seen domains cannot be stored, e.g., due to data protection regulations, the above becomes a challenging continual learning problem. To this end, we propose to use generative feature-driven image replay in conjunction with a dual-purpose discriminator that not only enables the generation of images with realistic features for replay, but also promotes feature alignment during domain adaptation. We evaluate our approach extensively on a sequence of three histopathological datasets for tissue-type classification, achieving state-of-the-art results. We present detailed ablation experiments studying our proposed method components and demonstrate a possible use-case of our continual UDA method for an unsupervised patch-based segmentation task given high-resolution tissue images.