On the Brittle Foundations of ReAct Prompting for Agentic Large Language Models

TL;DR

This paper critically examines ReAct prompting for large language models, revealing that its performance gains are primarily due to prompt similarity and retrieval effects rather than genuine reasoning improvements.

Contribution

The study systematically analyzes ReAct prompting, demonstrating that its effectiveness is largely attributable to prompt design and exemplar similarity, not enhanced reasoning capabilities.

Findings

Performance is minimally affected by reasoning trace content.

Similarity between input examples and queries drives LLM performance.

ReAct's perceived reasoning ability is due to exemplar-query similarity.

Abstract

The reasoning abilities of Large Language Models (LLMs) remain a topic of debate. Some methods such as ReAct-based prompting, have gained popularity for claiming to enhance sequential decision-making abilities of agentic LLMs. However, it is unclear what is the source of improvement in LLM reasoning with ReAct based prompting. In this paper we examine these claims of ReAct based prompting in improving agentic LLMs for sequential decision-making. By introducing systematic variations to the input prompt we perform a sensitivity analysis along the claims of ReAct and find that the performance is minimally influenced by the "interleaving reasoning trace with action execution" or the content of the generated reasoning traces in ReAct, contrary to original claims and common usage. Instead, the performance of LLMs is driven by the similarity between input example tasks and queries, implicitly forcing the prompt designer to provide instance-specific examples which significantly increases the cognitive burden on the human. Our investigation shows that the perceived reasoning abilities of LLMs stem from the exemplar-query similarity and approximate retrieval rather than any inherent reasoning abilities.