WHEN TO ACT, WHEN TO WAIT: Modeling the Intent-Action Alignment Problem in Dialogue

TL;DR

This paper introduces STORM, a framework for modeling the development of understanding in dialogue systems, addressing the critical timing of when a user’s expression is ready for system action, and reveals that partial transparency can sometimes outperform full clarity.

Contribution

The paper formalizes the Intent-Action Alignment Problem, models asymmetric information dynamics in dialogue, and proposes evaluation metrics for internal cognitive states and task performance.

Findings

Moderate uncertainty (40-60%) can outperform full transparency in some scenarios.

Model-specific patterns suggest reconsidering the level of information completeness.

Systematic analysis of how collaborative understanding evolves in dialogue.

Abstract

Dialogue systems often fail when user utterances are semantically complete yet lack the clarity and completeness required for appropriate system action. This mismatch arises because users frequently do not fully understand their own needs, while systems require precise intent definitions. This highlights the critical Intent-Action Alignment Problem: determining when an expression is not just understood, but truly ready for a system to act upon. We present STORM, a framework modeling asymmetric information dynamics through conversations between UserLLM (full internal access) and AgentLLM (observable behavior only). STORM produces annotated corpora capturing trajectories of expression phrasing and latent cognitive transitions, enabling systematic analysis of how collaborative understanding develops. Our contributions include: (1) formalizing asymmetric information processing in dialogue systems; (2) modeling intent formation tracking collaborative understanding evolution; and (3) evaluation metrics measuring internal cognitive improvements alongside task performance. Experiments across four language models reveal that moderate uncertainty (40-60%) can outperform complete transparency in certain scenarios, with model-specific patterns suggesting reconsideration of optimal information completeness in human-AI collaboration. These findings contribute to understanding asymmetric reasoning dynamics and inform uncertainty-calibrated dialogue system design.