A Kernel-Based View of Language Model Fine-Tuning

TL;DR

This paper explores how the Neural Tangent Kernel (NTK) can theoretically explain the success of fine-tuning large pre-trained language models, especially in low-data NLP tasks, and validates this with extensive experiments.

Contribution

It extends NTK theory to Adam optimizer and demonstrates its applicability to language model fine-tuning, providing a new theoretical perspective.

Findings

NTK describes fine-tuning dynamics in NLP tasks

Prompting often induces kernel-based fine-tuning behavior

Kernel view explains success of parameter-efficient methods

Abstract

It has become standard to solve NLP tasks by fine-tuning pre-trained language models (LMs), especially in low-data settings. There is minimal theoretical understanding of empirical success, e.g., why fine-tuning a model with $10^8$ or more parameters on a couple dozen training points does not result in overfitting. We investigate whether the Neural Tangent Kernel (NTK) - which originated as a model to study the gradient descent dynamics of infinitely wide networks with suitable random initialization - describes fine-tuning of pre-trained LMs. This study was inspired by the decent performance of NTK for computer vision tasks (Wei et al., 2022). We extend the NTK formalism to Adam and use Tensor Programs (Yang, 2020) to characterize conditions under which the NTK lens may describe fine-tuning updates to pre-trained language models. Extensive experiments on 14 NLP tasks validate our theory and show that formulating the downstream task as a masked word prediction problem through prompting often induces kernel-based dynamics during fine-tuning. Finally, we use this kernel view to propose an explanation for the success of parameter-efficient subspace-based fine-tuning methods.