Generalization Bounds for Deep Transfer Learning Using Majority Predictor Accuracy

TL;DR

This paper introduces new generalization bounds for deep transfer learning models based on majority predictor accuracy, which can be efficiently computed and used as a practical transferability measure validated by experiments.

Contribution

It proposes a novel theoretical framework linking majority predictor accuracy to generalization bounds and demonstrates its practical utility in transfer learning.

Findings

Majority predictor accuracy correlates with transferability.

The bounds are computationally efficient to evaluate.

Experiments validate the use of majority predictor accuracy as a transferability measure.

Abstract

We analyze new generalization bounds for deep learning models trained by transfer learning from a source to a target task. Our bounds utilize a quantity called the majority predictor accuracy, which can be computed efficiently from data. We show that our theory is useful in practice since it implies that the majority predictor accuracy can be used as a transferability measure, a fact that is also validated by our experiments.