Feasibility and Transferability of Transfer Learning: A Mathematical Framework

TL;DR

This paper introduces a mathematical framework for transfer learning, analyzing its feasibility and transferability through an optimization perspective and a new transfer risk concept, supported by empirical validation.

Contribution

It provides the first formal mathematical formulation of transfer learning and introduces transfer risk as a novel measure for transferability assessment.

Findings

Mathematical formulation of transfer learning as an optimization problem

Introduction of transfer risk to evaluate transferability

Empirical validation on Office-31 dataset showing benefits of transfer risk

Abstract

Transfer learning is an emerging and popular paradigm for utilizing existing knowledge from previous learning tasks to improve the performance of new ones. Despite its numerous empirical successes, theoretical analysis for transfer learning is limited. In this paper we build for the first time, to the best of our knowledge, a mathematical framework for the general procedure of transfer learning. Our unique reformulation of transfer learning as an optimization problem allows for the first time, analysis of its feasibility. Additionally, we propose a novel concept of transfer risk to evaluate transferability of transfer learning. Our numerical studies using the Office-31 dataset demonstrate the potential and benefits of incorporating transfer risk in the evaluation of transfer learning performance.