Symbol tuning improves in-context learning in language models

TL;DR

Symbol tuning, which replaces natural language labels with arbitrary symbols during finetuning, enhances language models' in-context learning, robustness, and reasoning abilities, especially on unseen tasks and when handling label variations.

Contribution

The paper introduces symbol tuning as a novel finetuning method that improves in-context learning and reasoning in large language models.

Findings

Boosts performance on unseen in-context tasks

Enhances robustness to underspecified prompts

Improves ability to follow flipped labels in-context

Abstract

We present symbol tuning - finetuning language models on in-context input-label pairs where natural language labels (e.g., "positive/negative sentiment") are replaced with arbitrary symbols (e.g., "foo/bar"). Symbol tuning leverages the intuition that when a model cannot use instructions or natural language labels to figure out a task, it must instead do so by learning the input-label mappings. We experiment with symbol tuning across Flan-PaLM models up to 540B parameters and observe benefits across various settings. First, symbol tuning boosts performance on unseen in-context learning tasks and is much more robust to underspecified prompts, such as those without instructions or without natural language labels. Second, symbol-tuned models are much stronger at algorithmic reasoning tasks, with up to 18.2% better performance on the List Functions benchmark and up to 15.3% better performance on the Simple Turing Concepts benchmark. Finally, symbol-tuned models show large improvements in following flipped-labels presented in-context, meaning that they are more capable of using in-context information to override prior semantic knowledge.