The Mystery of the Pathological Path-star Task for Language Models

TL;DR

The paper investigates the limitations of language models on the path-star task, introduces a regularization method to improve performance, and demonstrates the task's theoretical solvability and potential for encoder-only models.

Contribution

It reveals the challenges of language models on the path-star task, proposes a novel regularization technique, and provides theoretical proofs of the task's solvability.

Findings

Regularization improves model performance on the task.

The task is theoretically solvable with proper training.

Encoder-only models can solve the task under certain conditions.

Abstract

The recently introduced path-star task is a minimal task designed to exemplify limitations to the abilities of language models (Bachmann and Nagarajan, 2024). It involves a path-star graph where multiple arms radiate from a single starting node and each node is unique. Given the start node and a specified target node that ends an arm, the task is to generate the arm containing that target node. This is straightforward for a human but surprisingly difficult for language models, which did not outperform the random baseline. The authors hypothesized this is due to a deficiency in teacher-forcing and the next-token prediction paradigm. We demonstrate the task is learnable using teacher-forcing in alternative settings and that the issue is partially due to representation. We introduce a regularization method using structured samples of the same graph but with differing target nodes, improving results across a variety of model types. We provide RASP proofs showing the task is theoretically solvable. Finally, we find settings where an encoder-only model can consistently solve the task.