You Only Fine-tune Once: Many-Shot In-Context Fine-Tuning for Large Language Models

TL;DR

This paper introduces Many-Shot In-Context Fine-Tuning (ManyICL), a method that enhances large language models' performance on multiple tasks by treating in-context examples as supervised targets, narrowing the gap with dedicated fine-tuning.

Contribution

The paper proposes ManyICL, a novel training objective and approach that improves multi-task learning in LLMs by leveraging in-context examples as supervised signals, reducing the need for separate fine-tuning.

Findings

ManyICL outperforms zero/few-shot fine-tuning on diverse tasks.

It approaches the performance of dedicated fine-tuning.

It mitigates catastrophic forgetting in multi-task settings.

Abstract

Large language models (LLMs) possess a remarkable ability to perform in-context learning (ICL), which enables them to handle multiple downstream tasks simultaneously without requiring task-specific fine-tuning. Recent studies have shown that even moderately sized LLMs, such as Mistral 7B, Gemma 7B and Llama-3 8B, can achieve ICL through few-shot in-context fine-tuning of all tasks at once. However, this approach still lags behind dedicated fine-tuning, where a separate model is trained for each individual task. In this paper, we propose a novel approach, Many-Shot In-Context Fine-tuning (ManyICL), which significantly narrows this performance gap by extending the principles of ICL to a many-shot setting. To unlock the full potential of ManyICL and address the inherent inefficiency of processing long sequences with numerous in-context examples, we propose a novel training objective. Instead of solely predicting the final answer, our approach treats every answer within the context as a supervised training target. This effectively shifts the role of many-shot examples from prompts to targets for autoregressive learning. Through extensive experiments on diverse downstream tasks, including classification, summarization, question answering, natural language inference, and math, we demonstrate that ManyICL substantially outperforms zero/few-shot fine-tuning and approaches the performance of dedicated fine-tuning. Furthermore, ManyICL significantly mitigates catastrophic forgetting issues observed in zero/few-shot fine-tuning. The code will be made publicly available upon publication.