The Inherent Limits of Pretrained LLMs: The Unexpected Convergence of Instruction Tuning and In-Context Learning Capabilities

TL;DR

This paper investigates the capabilities of large language models, revealing that instruction tuning does not fundamentally change their abilities but rather constrains them within limits set by pretraining data, linking in-context learning and instruction tuning.

Contribution

The study demonstrates that instruction-tuned models' performance is closely linked to their base models' in-context abilities, highlighting the limits imposed by pretraining data on both.

Findings

Instruction tuning performance correlates with base model in-context performance.

Pretraining data limits the tasks that instruction-tuned models can solve.

Instruction tuning does not fundamentally alter the core capabilities of LLMs.

Abstract

Large Language Models (LLMs), trained on extensive web-scale corpora, have demonstrated remarkable abilities across diverse tasks, especially as they are scaled up. Nevertheless, even state-of-the-art models struggle in certain cases, sometimes failing at problems solvable by young children, indicating that traditional notions of task complexity are insufficient for explaining LLM capabilities. However, exploring LLM capabilities is complicated by the fact that most widely-used models are also "instruction-tuned" to respond appropriately to prompts. With the goal of disentangling the factors influencing LLM performance, we investigate whether instruction-tuned models possess fundamentally different capabilities from base models that are prompted using in-context examples. Through extensive experiments across various model families, scales and task types, which included instruction tuning 90 different LLMs, we demonstrate that the performance of instruction-tuned models is significantly correlated with the in-context performance of their base counterparts. By clarifying what instruction-tuning contributes, we extend prior research into in-context learning, which suggests that base models use priors from pretraining data to solve tasks. Specifically, we extend this understanding to instruction-tuned models, suggesting that their pretraining data similarly sets a limiting boundary on the tasks they can solve, with the added influence of the instruction-tuning dataset.