Systematic Generalization in Language Models Scales with Information Entropy

TL;DR

This paper links systematic generalization in language models to the entropy of training data, showing that model performance scales with entropy and suggesting entropy as a measure for assessing and improving generalization.

Contribution

It introduces a formal framework for measuring entropy in sequence tasks and demonstrates its correlation with model performance across different architectures.

Findings

Model performance improves with higher data entropy.

Success at high entropy is achievable without built-in priors.

Low entropy performance can gauge progress in systematic generalization.

Abstract

Systematic generalization remains challenging for current language models, which are known to be both sensitive to semantically similar permutations of the input and to struggle with known concepts presented in novel contexts. Although benchmarks exist for assessing compositional behavior, it is unclear how to measure the difficulty of a systematic generalization problem. In this work, we show how one aspect of systematic generalization can be described by the entropy of the distribution of component parts in the training data. We formalize a framework for measuring entropy in a sequence-to-sequence task and find that the performance of popular model architectures scales with the entropy. Our work connects systematic generalization to information efficiency, and our results indicate that success at high entropy can be achieved even without built-in priors, and that success at low entropy can serve as a target for assessing progress towards robust systematic generalization.