When Models Know More Than They Say: Probing Analogical Reasoning in LLMs

TL;DR

This paper investigates the ability of large language models to perform analogical reasoning, revealing that internal representations often contain more knowledge than is accessible through prompting, especially for rhetorical analogies.

Contribution

It compares model representations and prompted performance, highlighting task-dependent limitations in how models access latent analogical reasoning capabilities.

Findings

Probing outperforms prompting for rhetorical analogies in open-source models.

Prompting and probing perform similarly and poorly for narrative analogies.

Internal representations contain more analogical knowledge than what prompting reveals.

Abstract

Analogical reasoning is a core cognitive faculty essential for narrative understanding. While LLMs perform well when surface and structural cues align, they struggle in cases where an analogy is not apparent on the surface but requires latent information, suggesting limitations in abstraction and generalisation. In this paper we compare a model's probed representations with its prompted performance at detecting narrative analogies, revealing an asymmetry: for rhetorical analogies, probing significantly outperforms prompting in open-source models, while for narrative analogies, they achieve a similar (low) performance. This suggests that the relationship between internal representations and prompted behavior is task-dependent and may reflect limitations in how prompting accesses available information.