State Design Matters: How Representations Shape Dynamic Reasoning in Large Language Models

TL;DR

This paper investigates how different state representations—such as granularity, structure, and spatial grounding—affect the dynamic reasoning capabilities of large language models in sequential decision-making tasks.

Contribution

It systematically evaluates the impact of various state encoding strategies on LLM performance, highlighting the importance of design choices in state representation.

Findings

Trajectory summarisation enhances stability and performance.

Natural language representations are most robust across models.

Text-based spatial encodings outperform image inputs.

Abstract

As large language models (LLMs) move from static reasoning tasks toward dynamic environments, their success depends on the ability to navigate and respond to an environment that changes as they interact at inference time. An underexplored factor in these settings is the representation of the state. Holding model parameters fixed, we systematically vary three key aspects: (1) state granularity (long form versus summary), (2) structure (natural language versus symbolic), and (3) spatial grounding (text-only versus images or textual map encodings) across sequential decision-making benchmarks. We find that trajectory summarisation improves performance by reducing noise and stabilising long-horizon reasoning. Second, natural language representations are the most robust across models, whereas structured encodings help mainly for models with strong code or structured output priors, such as JSON schemas. Third, while image-inputs show some benefit, text-based spatial encodings prove most effective. This advantage stems not from the spatial information itself, but from the act of construction, which compels the model to perform the spatial reasoning that static input does not elicit. Overall, we demonstrate that design choices for representing state are a decisive factor in performance, distinct from the availability of information itself. We note, however, that even with improved representations, current LLMs and VLMs remain brittle over long horizons, particularly when they must synthesise information to manage multiple subtasks to reach a goal.