Transfer Learning for Algorithm Recommendation

TL;DR

This paper explores the application of transfer learning at the meta-level within meta-learning, specifically for algorithm recommendation, to assess how effectively knowledge can be transferred across related datasets.

Contribution

It investigates the novel use of transfer learning in meta-learning for algorithm recommendation, focusing on transferring meta-knowledge between similar datasets.

Findings

Transfer learning improves meta-model performance on new datasets.

Meta-knowledge transfer accelerates learning in algorithm recommendation.

Transferability of meta-knowledge varies with dataset similarity.

Abstract

Meta-Learning is a subarea of Machine Learning that aims to take advantage of prior knowledge to learn faster and with fewer data [1]. There are different scenarios where meta-learning can be applied, and one of the most common is algorithm recommendation, where previous experience on applying machine learning algorithms for several datasets can be used to learn which algorithm, from a set of options, would be more suitable for a new dataset [2]. Perhaps the most popular form of meta-learning is transfer learning, which consists of transferring knowledge acquired by a machine learning algorithm in a previous learning task to increase its performance faster in another and similar task [3]. Transfer Learning has been widely applied in a variety of complex tasks such as image classification, machine translation and, speech recognition, achieving remarkable results [4,5,6,7,8]. Although transfer learning is very used in traditional or base-learning, it is still unknown if it is useful in a meta-learning setup. For that purpose, in this paper, we investigate the effects of transferring knowledge in the meta-level instead of base-level. Thus, we train a neural network on meta-datasets related to algorithm recommendation, and then using transfer learning, we reuse the knowledge learned by the neural network in other similar datasets from the same domain, to verify how transferable is the acquired meta-knowledge.