Controlling Long-Horizon Behavior in Language Model Agents with Explicit State Dynamics

TL;DR

This paper explores how explicit affective state dynamics can improve long-term coherence and control in language model agents during extended dialogues by maintaining and updating an external emotional state.

Contribution

It introduces an affective subsystem with Valence-Arousal-Dominance dynamics that enhances temporal coherence without altering model parameters.

Findings

State persistence enables delayed responses and recovery.

Second-order dynamics increase affective inertia and hysteresis.

Stateless agents lack coherent trajectories.

Abstract

Large language model (LLM) agents often exhibit abrupt shifts in tone and persona during extended interaction, reflecting the absence of explicit temporal structure governing agent-level state. While prior work emphasizes turn-local sentiment or static emotion classification, the role of explicit affective dynamics in shaping long-horizon agent behavior remains underexplored. This work investigates whether imposing dynamical structure on an external affective state can induce temporal coherence and controlled recovery in multi-turn dialogue. We introduce an agent-level affective subsystem that maintains a continuous Valence-Arousal-Dominance (VAD) state external to the language model and governed by first- and second-order update rules. Instantaneous affective signals are extracted using a fixed, memoryless estimator and integrated over time via exponential smoothing or momentum-based dynamics. The resulting affective state is injected back into generation without modifying model parameters. Using a fixed 25-turn dialogue protocol, we compare stateless, first-order, and second-order affective dynamics. Stateless agents fail to exhibit coherent trajectories or recovery, while state persistence enables delayed responses and reliable recovery. Second-order dynamics introduce affective inertia and hysteresis that increase with momentum, revealing a trade-off between stability and responsiveness.