On the Emergence and Test-Time Use of Structural Information in Large Language Models

TL;DR

This paper investigates how large language models learn and utilize structural information, revealing that such learning correlates with complex reasoning but test-time compositional generation remains limited.

Contribution

It introduces a controlled dataset and empirically analyzes the emergence and use of structural information in language models.

Findings

Structural learning correlates with complex reasoning tasks

Test-time compositional generation ability remains limited

Structural information emergence depends on task complexity

Abstract

Learning structural information from observational data is central to producing new knowledge outside the training corpus. This holds for mechanistic understanding in scientific discovery as well as flexible test-time compositional generation. We thus study how language models learn abstract structures and utilize the learnt structural information at test-time. To ensure a controlled setup, we design a natural language dataset based on linguistic structural transformations. We empirically show that the emergence of learning structural information correlates with complex reasoning tasks, and that the ability to perform test-time compositional generation remains limited.