TAKT: Target-Aware Knowledge Transfer for Whole Slide Image Classification

TL;DR

This paper introduces TAKT, a target-aware knowledge transfer framework for whole slide image classification that leverages a teacher-student paradigm and optimal transport to improve transfer learning across domains.

Contribution

The paper proposes a novel target-aware knowledge transfer method with a feature alignment module to address domain shift in whole slide image classification.

Findings

Models with TAKT outperform training from scratch.

Achieves state-of-the-art results on multiple datasets.

Effective in handling domain shift and task discrepancy.

Abstract

Transferring knowledge from a source domain to a target domain can be crucial for whole slide image classification, since the number of samples in a dataset is often limited due to high annotation costs. However, domain shift and task discrepancy between datasets can hinder effective knowledge transfer. In this paper, we propose a Target-Aware Knowledge Transfer framework, employing a teacher-student paradigm. Our framework enables the teacher model to learn common knowledge from the source and target domains by actively incorporating unlabelled target images into the training of the teacher model. The teacher bag features are subsequently adapted to supervise the training of the student model on the target domain. Despite incorporating the target features during training, the teacher model tends to overlook them under the inherent domain shift and task discrepancy. To alleviate this, we introduce a target-aware feature alignment module to establish a transferable latent relationship between the source and target features by solving the optimal transport problem. Experimental results show that models employing knowledge transfer outperform those trained from scratch, and our method achieves state-of-the-art performance among other knowledge transfer methods on various datasets, including TCGA-RCC, TCGA-NSCLC, and Camelyon16.