Fine-Tuning is Subgraph Search: A New Lens on Learning Dynamics

TL;DR

This paper introduces a novel fine-tuning method called circuit-tuning that models learning as subgraph search within a neural network, offering insights into learning dynamics and improving task-specific performance.

Contribution

It proposes a new analytical framework for understanding fine-tuning as subgraph search, validated through experiments and analysis of learning dynamics.

Findings

Circuit-tuning balances task performance and general capabilities.

Learning dynamics can be understood through subgraph optimization.

The method offers new insights into neural network training mechanisms.

Abstract

The study of mechanistic interpretability aims to reverse-engineer a model to explain its behaviors. While recent studies have focused on the static mechanism of a certain behavior, the learning dynamics inside a model remain to be explored. In this work, we develop a fine-tuning method for analyzing the mechanism behind learning. Inspired by the concept of intrinsic dimension, we view a model as a computational graph with redundancy for a specific task, and treat the fine-tuning process as a search for and optimization of a subgraph within this graph. Based on this hypothesis, we propose circuit-tuning, an algorithm that iteratively builds the subgraph for a specific task and updates the relevant parameters in a heuristic way. We first validate our hypothesis through a carefully designed experiment and provide a detailed analysis of the learning dynamics during fine-tuning. Subsequently, we conduct experiments on more complex tasks, demonstrating that circuit-tuning could strike a balance between the performance on the target task and the general capabilities. Our work offers a new analytical method for the dynamics of fine-tuning, provides new findings on the mechanisms behind the training process, and inspires the design of superior algorithms for the training of neural networks.