Task-Agnostic Continual Learning for Chest Radiograph Classification

TL;DR

This paper introduces CARL-XRay, a task-agnostic continual learning method for chest radiograph classification that maintains high performance without retraining on previous data, enabling practical clinical deployment.

Contribution

It proposes a novel adapter-based routing strategy with a latent task selector and experience replay, supporting stable, task-agnostic continual learning for chest X-ray classification.

Findings

Outperforms joint training in task-unknown settings

Achieves 75.0% routing accuracy and 0.74 AUROC in oracle setting

Uses fewer parameters while maintaining competitive performance

Abstract

Clinical deployment of chest radiograph classifiers requires models that can be updated as new datasets become available without retraining on previously ob- served data or degrading validated performance. We study, for the first time, a task-incremental continual learning setting for chest radiograph classification, in which heterogeneous chest X-ray datasets arrive sequentially and task identifiers are unavailable at inference. We propose a continual adapter-based routing learning strategy for Chest X-rays (CARL-XRay) that maintains a fixed high-capacity backbone and incrementally allocates lightweight task-specific adapters and classifier heads. A latent task selector operates on task-adapted features and leverages both current and historical context preserved through compact prototypes and feature-level experience replay. This design supports stable task identification and adaptation across sequential updates while avoiding raw-image storage. Experiments on large-scale public chest radiograph datasets demonstrate robust performance retention and reliable task-aware inference under continual dataset ingestion. CARL-XRay outperforms joint training under task-unknown deployment, achieving higher routing accuracy (75.0\% vs.\ 62.5\%), while maintaining competitive diagnostic performance with AUROC of 0.74 in the oracle setting with ground-truth task identity and 0.75 under task-unknown inference, using significantly fewer trainable parameters. Finally, the proposed framework provides a practical alternative to joint training and repeated full retraining in continual clinical deployment.