Mitigating Conversational Inertia in Multi-Turn Agents

TL;DR

This paper identifies conversational inertia in multi-turn language models, analyzes its impact, and proposes a preference learning framework with context management strategies to improve agent performance.

Contribution

It introduces the concept of conversational inertia, analyzes its effects, and develops Context Preference Learning to mitigate inertia and enhance multi-turn agent capabilities.

Findings

Reducing conversational inertia improves agent performance across multiple environments.

Longer contexts increase inertia, hindering exploration in multi-turn interactions.

Context Preference Learning effectively balances exploration and exploitation in language models.

Abstract

Large language models excel as few-shot learners when provided with appropriate demonstrations, yet this strength becomes problematic in multiturn agent scenarios, where LLMs erroneously mimic their own previous responses as few-shot examples. Through attention analysis, we identify conversational inertia, a phenomenon where models exhibit strong diagonal attention to previous responses, which is associated with imitation bias that constrains exploration. This reveals a tension when transforming few-shot LLMs into agents: longer context enriches environmental feedback for exploitation, yet also amplifies conversational inertia that undermines exploration. Our key insight is that for identical states, actions generated with longer contexts exhibit stronger inertia than those with shorter contexts, enabling construction of preference pairs without environment rewards. Based on this, we propose Context Preference Learning to calibrate model preferences to favor low-inertia responses over highinertia ones. We further provide context management strategies at inference time to balance exploration and exploitation. Experimental results across eight agentic environments and one deep research scenario validate that our framework reduces conversational inertia and achieves performance improvements.