Lessons from Studying Two-Hop Latent Reasoning

TL;DR

This paper investigates the latent two-hop reasoning capabilities of large language models using a controlled synthetic fact setting, revealing nuanced strengths and limitations in their reasoning abilities.

Contribution

It introduces a controlled experimental framework to definitively assess latent two-hop reasoning in LLMs, highlighting the importance of experimental design in evaluating reasoning capabilities.

Findings

Models can perform two-hop reasoning when combining synthetic and natural facts.

Performance varies depending on the nature of facts involved, indicating nuanced reasoning abilities.

Results caution against misinterpreting successes or failures due to memorization or experimental artifacts.

Abstract

Large language models can use chain-of-thought (CoT) to externalize reasoning, potentially enabling oversight of capable LLM agents. Prior work has shown that models struggle at two-hop question-answering without CoT. This capability is so basic that if it was a fundamental limitation, it would imply that many complex agentic tasks would similarly require CoT. We investigate LLM latent reasoning capabilities using two-hop question answering as a case study. Previous work on the gap between latent and externalized two-hop reasoning produced mixed evidence with inconclusive results. In this paper, we introduce a controlled setting for investigating two-hop reasoning in LLMs, where a positive result provides definitive evidence for latent reasoning. We fine-tune LLMs (including Llama 3 8B and GPT-4o) on synthetic facts and test two-hop reasoning over these facts. By using synthetic facts, we rule out memorization and reasoning shortcuts as explanations for two-hop performance. We observe a nuanced picture: Models fail to compose two synthetic facts, but can succeed when one fact is synthetic and the other is natural. These results demonstrate that LLMs are undeniably capable of latent two-hop reasoning, although it remains unclear how this ability scales with model size. Finally, we highlight a lesson for researchers studying LLM reasoning: when drawing conclusions about LLM latent reasoning, one must be careful to avoid both spurious successes (that stem from memorization and reasoning shortcuts) and spurious failures (that may stem from artificial experimental setups, divorced from training setups of frontier LLMs).