Modular Adaptation for Cross-Domain Few-Shot Learning

TL;DR

This paper introduces a modular adaptation approach that sequences multiple adaptation methods and dynamically configures them for cross-domain few-shot learning, significantly improving performance on diverse benchmarks.

Contribution

It proposes a novel modular adaptation pipeline that customizes adaptation strategies for different tasks, enhancing cross-domain few-shot learning performance.

Findings

Achieves 3.1% higher accuracy over baselines.

Creates a new high-way 100-way k-shot benchmark.

Demonstrates the effectiveness of modular adaptation in diverse domains.

Abstract

Adapting pre-trained representations has become the go-to recipe for learning new downstream tasks with limited examples. While literature has demonstrated great successes via representation learning, in this work, we show that substantial performance improvement of downstream tasks can also be achieved by appropriate designs of the adaptation process. Specifically, we propose a modular adaptation method that selectively performs multiple state-of-the-art (SOTA) adaptation methods in sequence. As different downstream tasks may require different types of adaptation, our modular adaptation enables the dynamic configuration of the most suitable modules based on the downstream task. Moreover, as an extension to existing cross-domain 5-way k-shot benchmarks (e.g., miniImageNet -> CUB), we create a new high-way (~100) k-shot benchmark with data from 10 different datasets. This benchmark provides a diverse set of domains and allows the use of stronger representations learned from ImageNet. Experimental results show that by customizing adaptation process towards downstream tasks, our modular adaptation pipeline (MAP) improves 3.1% in 5-shot classification accuracy over baselines of finetuning and Prototypical Networks.