Evaluating the Values of Sources in Transfer Learning

TL;DR

This paper introduces SEAL-Shap, a framework that quantifies the usefulness of sources in transfer learning for NLP, enabling better source selection and understanding of source-target similarities.

Contribution

It develops an efficient source valuation method based on Shapley values for transfer learning, improving source selection and interpretability.

Findings

SEAL-Shap effectively identifies useful transfer sources.

Source values align with intuitive source-target similarities.

The framework improves transfer learning performance.

Abstract

Transfer learning that adapts a model trained on data-rich sources to low-resource targets has been widely applied in natural language processing (NLP). However, when training a transfer model over multiple sources, not every source is equally useful for the target. To better transfer a model, it is essential to understand the values of the sources. In this paper, we develop SEAL-Shap, an efficient source valuation framework for quantifying the usefulness of the sources (e.g., domains/languages) in transfer learning based on the Shapley value method. Experiments and comprehensive analyses on both cross-domain and cross-lingual transfers demonstrate that our framework is not only effective in choosing useful transfer sources but also the source values match the intuitive source-target similarity.