Taxonomy Adaptive Cross-Domain Adaptation in Medical Imaging via Optimization Trajectory Distillation

TL;DR

This paper introduces a novel method called optimization trajectory distillation to improve unsupervised domain adaptation in medical imaging, especially when dealing with taxonomic inconsistencies and limited annotations, by leveraging gradient space properties.

Contribution

It presents a unified approach that considers learning dynamics and gradient low-rank structure to address domain shifts and class inconsistencies in medical image analysis.

Findings

Outperforms previous methods in various clinical and open-world tasks.

Effectively handles taxonomic inconsistency and limited annotations.

Demonstrates significant improvements in adaptation accuracy.

Abstract

The success of automated medical image analysis depends on large-scale and expert-annotated training sets. Unsupervised domain adaptation (UDA) has been raised as a promising approach to alleviate the burden of labeled data collection. However, they generally operate under the closed-set adaptation setting assuming an identical label set between the source and target domains, which is over-restrictive in clinical practice where new classes commonly exist across datasets due to taxonomic inconsistency. While several methods have been presented to tackle both domain shifts and incoherent label sets, none of them take into account the common characteristics of the two issues and consider the learning dynamics along network training. In this work, we propose optimization trajectory distillation, a unified approach to address the two technical challenges from a new perspective. It exploits the low-rank nature of gradient space and devises a dual-stream distillation algorithm to regularize the learning dynamics of insufficiently annotated domain and classes with the external guidance obtained from reliable sources. Our approach resolves the issue of inadequate navigation along network optimization, which is the major obstacle in the taxonomy adaptive cross-domain adaptation scenario. We evaluate the proposed method extensively on several tasks towards various endpoints with clinical and open-world significance. The results demonstrate its effectiveness and improvements over previous methods.