When & How to Transfer with Transfer Learning

TL;DR

This paper evaluates transfer learning in deep image tasks, analyzing its benefits, costs, and trade-offs, and provides guidelines for optimal use based on experimental results.

Contribution

It offers a comprehensive experimental analysis of transfer learning trade-offs and proposes practical guidelines for its application in various scenarios.

Findings

Cheap feature extraction is preferable in some cases.

Fine-tuning can be worth the cost in certain situations.

Transfer learning impacts performance, environmental footprint, and human effort.

Abstract

In deep learning, transfer learning (TL) has become the de facto approach when dealing with image related tasks. Visual features learnt for one task have been shown to be reusable for other tasks, improving performance significantly. By reusing deep representations, TL enables the use of deep models in domains with limited data availability, limited computational resources and/or limited access to human experts. Domains which include the vast majority of real-life applications. This paper conducts an experimental evaluation of TL, exploring its trade-offs with respect to performance, environmental footprint, human hours and computational requirements. Results highlight the cases were a cheap feature extraction approach is preferable, and the situations where an expensive fine-tuning effort may be worth the added cost. Finally, a set of guidelines on the use of TL are proposed.