Class-Incremental Continual Learning for General Purpose Healthcare Models

TL;DR

This paper explores the use of continual learning models in healthcare imaging to adapt to changing data without forgetting previous knowledge, demonstrating their potential for general-purpose medical AI across diverse medical specialties.

Contribution

It evaluates multiple continual learning approaches across various medical imaging datasets, showing that a single model can learn sequentially from different specialties effectively.

Findings

Models can learn new medical tasks without performance loss on old tasks.

Continual learning approaches achieve comparable results to naive fine-tuning.

Feasibility of sharing models across different medical institutions and specialties.

Abstract

Healthcare clinics regularly encounter dynamic data that changes due to variations in patient populations, treatment policies, medical devices, and emerging disease patterns. Deep learning models can suffer from catastrophic forgetting when fine-tuned in such scenarios, causing poor performance on previously learned tasks. Continual learning allows learning on new tasks without performance drop on previous tasks. In this work, we investigate the performance of continual learning models on four different medical imaging scenarios involving ten classification datasets from diverse modalities, clinical specialties, and hospitals. We implement various continual learning approaches and evaluate their performance in these scenarios. Our results demonstrate that a single model can sequentially learn new tasks from different specialties and achieve comparable performance to naive methods. These findings indicate the feasibility of recycling or sharing models across the same or different medical specialties, offering another step towards the development of general-purpose medical imaging AI that can be shared across institutions.