SHiFT: An Efficient, Flexible Search Engine for Transfer Learning

TL;DR

SHiFT is a novel search engine designed for transfer learning models that efficiently identifies suitable models for specific tasks using a flexible query language and cost-based decision making, improving model selection processes.

Contribution

The paper introduces SHiFT, the first flexible, task-aware model search engine for transfer learning that combines a custom query language with a cost-based decision framework.

Findings

Hybrid search strategies outperform single methods.

SHiFT's query language enables flexible model selection.

Incremental query execution improves efficiency.

Abstract

Transfer learning can be seen as a data- and compute-efficient alternative to training models from scratch. The emergence of rich model repositories, such as TensorFlow Hub, enables practitioners and researchers to unleash the potential of these models across a wide range of downstream tasks. As these repositories keep growing exponentially, efficiently selecting a good model for the task at hand becomes paramount. By carefully comparing various selection and search strategies, we realize that no single method outperforms the others, and hybrid or mixed strategies can be beneficial. Therefore, we propose SHiFT, the first downstream task-aware, flexible, and efficient model search engine for transfer learning. These properties are enabled by a custom query language SHiFT-QL together with a cost-based decision maker, which we empirically validate. Motivated by the iterative nature of machine learning development, we further support efficient incremental executions of our queries, which requires a careful implementation when jointly used with our optimizations.