BeST -- A Novel Source Selection Metric for Transfer Learning

TL;DR

This paper introduces BeST, a new task-similarity metric for transfer learning that efficiently identifies the most suitable source models for a target task, saving computational resources.

Contribution

We propose a novel, quick-to-compute similarity metric (BeST) that predicts the transferability of source models without extensive training.

Findings

BeST accurately predicts the best source models across datasets.

The metric reduces computational costs before transfer learning.

Experimental results show consistent performance improvements.

Abstract

One of the most fundamental, and yet relatively less explored, goals in transfer learning is the efficient means of selecting top candidates from a large number of previously trained models (optimized for various "source" tasks) that would perform the best for a new "target" task with a limited amount of data. In this paper, we undertake this goal by developing a novel task-similarity metric (BeST) and an associated method that consistently performs well in identifying the most transferrable source(s) for a given task. In particular, our design employs an innovative quantization-level optimization procedure in the context of classification tasks that yields a measure of similarity between a source model and the given target data. The procedure uses a concept similar to early stopping (usually implemented to train deep neural networks (DNNs) to ensure generalization) to derive a function that approximates the transfer learning mapping without training. The advantage of our metric is that it can be quickly computed to identify the top candidate(s) for a given target task before a computationally intensive transfer operation (typically using DNNs) can be implemented between the selected source and the target task. As such, our metric can provide significant computational savings for transfer learning from a selection of a large number of possible source models. Through extensive experimental evaluations, we establish that our metric performs well over different datasets and varying numbers of data samples.