P2L: Predicting Transfer Learning for Images and Semantic Relations

TL;DR

This paper introduces P2L, a method to predict the most suitable pre-trained model for transfer learning in image and semantic relation tasks, outperforming simple heuristics based on data size.

Contribution

The paper presents P2L, an efficient measure to accurately select the best source model for transfer learning across diverse domains, improving transfer success prediction.

Findings

P2L outperforms data size heuristic in selecting optimal transfer models.

P2L accurately predicts effective transfer learning models across 95 tasks.

Shared informational structure influences transfer success more than data size.

Abstract

Transfer learning enhances learning across tasks, by leveraging previously learned representations -- if they are properly chosen. We describe an efficient method to accurately estimate the appropriateness of a previously trained model for use in a new learning task. We use this measure, which we call "Predict To Learn" ("P2L"), in the two very different domains of images and semantic relations, where it predicts, from a set of "source" models, the one model most likely to produce effective transfer for training a given "target" model. We validate our approach thoroughly, by assembling a collection of candidate source models, then fine-tuning each candidate to perform each of a collection of target tasks, and finally measuring how well transfer has been enhanced. Across 95 tasks within multiple domains (images classification and semantic relations), the P2L approach was able to select the best transfer learning model on average, while the heuristic of choosing model trained with the largest data set selected the best model in only 55 cases. These results suggest that P2L captures important information in common between source and target tasks, and that this shared informational structure contributes to successful transfer learning more than simple data size.