Transfer Learning for Finetuning Large Language Models

TL;DR

This paper introduces a transfer learning approach for finetuning large language models, leveraging meta-learning to transfer knowledge from related tasks, which improves efficiency and effectiveness over traditional methods.

Contribution

It proposes a novel transfer learning method that uses meta-learning of performance and cost surrogate models, avoiding task-specific Bayesian optimization for better finetuning of large language models.

Findings

Transfer learning outperforms zero-shot and default finetuning.

Meta-learning surrogate models improves finetuning efficiency.

Method shows superior results on synthetic and real datasets.

Abstract

As the landscape of large language models expands, efficiently finetuning for specific tasks becomes increasingly crucial. At the same time, the landscape of parameter-efficient finetuning methods rapidly expands. Consequently, practitioners face a multitude of complex choices when searching for an optimal finetuning pipeline for large language models. To reduce the complexity for practitioners, we investigate transfer learning for finetuning large language models and aim to transfer knowledge about configurations from related finetuning tasks to a new task. In this work, we transfer learn finetuning by meta-learning performance and cost surrogate models for grey-box meta-optimization from a new meta-dataset. Counter-intuitively, we propose to rely only on transfer learning for new datasets. Thus, we do not use task-specific Bayesian optimization but prioritize knowledge transferred from related tasks over task-specific feedback. We evaluate our method on eight synthetic question-answer datasets and a meta-dataset consisting of 1,800 runs of finetuning Microsoft's Phi-3. Our transfer learning is superior to zero-shot, default finetuning, and meta-optimization baselines. Our results demonstrate the transferability of finetuning to adapt large language models more effectively.