Transformer See, Transformer Do: Copying as an Intermediate Step in Learning Analogical Reasoning

TL;DR

This paper demonstrates that transformer models trained with meta-learning on analogical reasoning tasks can generalize to new problems and datasets, especially when guided by copying tasks and heterogeneous data, revealing insights into their reasoning processes.

Contribution

The study introduces a meta-learning approach that enhances transformer models' ability to perform analogical reasoning and generalize, with interpretability analyses linking model behavior to an underlying algorithm.

Findings

Models learn to attend to informative problem elements through copying tasks.

Generalization improves with more heterogeneous training datasets.

The approach enables some transfer to composed transformations, but not entirely new ones.

Abstract

Analogical reasoning is a hallmark of human intelligence, enabling us to solve new problems by transferring knowledge from one situation to another. Yet, developing artificial intelligence systems capable of robust human-like analogical reasoning has proven difficult. In this work, we train transformers using Meta-Learning for Compositionality (MLC) on an analogical reasoning task (letter-string analogies) and assess their generalization capabilities. We find that letter-string analogies become learnable when guiding the models to attend to the most informative problem elements induced by including copying tasks in the training data. Furthermore, generalization to new alphabets becomes better when models are trained with more heterogeneous datasets, where our 3-layer encoder-decoder model outperforms most frontier models. The MLC approach also enables some generalization to compositions of trained transformations, but not to completely novel transformations. To understand how the model operates, we identify an algorithm that approximates the model's computations. We verify this using interpretability analyses and show that the model can be steered precisely according to expectations derived from the algorithm. Finally, we discuss implications of our findings for generalization capabilities of larger models and parallels to human analogical reasoning.