Welcome New Doctor: Continual Learning with Expert Consultation and Autoregressive Inference for Whole Slide Image Analysis

TL;DR

This paper introduces COSFormer, a Transformer-based continual learning framework that enables efficient, sequential analysis of whole slide images across multiple tasks and organs without revisiting past data.

Contribution

The study presents a novel continual learning method specifically designed for large-scale WSI analysis, reducing resource needs while maintaining high performance.

Findings

COSFormer outperforms existing continual learning methods in WSI tasks.

It demonstrates strong generalizability across seven organs and six tasks.

The framework effectively balances resource efficiency with high accuracy.

Abstract

Whole Slide Image (WSI) analysis, with its ability to reveal detailed tissue structures in magnified views, plays a crucial role in cancer diagnosis and prognosis. Due to their giga-sized nature, WSIs require substantial storage and computational resources for processing and training predictive models. With the rapid increase in WSIs used in clinics and hospitals, there is a growing need for a continual learning system that can efficiently process and adapt existing models to new tasks without retraining or fine-tuning on previous tasks. Such a system must balance resource efficiency with high performance. In this study, we introduce COSFormer, a Transformer-based continual learning framework tailored for multi-task WSI analysis. COSFormer is designed to learn sequentially from new tasks wile avoiding the need to revisit full historical datasets. We evaluate COSFormer on a sequence of seven WSI datasets covering seven organs and six WSI-related tasks under both class-incremental and task-incremental settings. The results demonstrate COSFormer's superior generalizability and effectiveness compared to existing continual learning frameworks, establishing it as a robust solution for continual WSI analysis in clinical applications.