ThoughtTrace: Understanding User Thoughts in Real-World LLM Interactions

TL;DR

ThoughtTrace introduces a large-scale dataset pairing human-AI conversations with users' self-reported thoughts, enabling better understanding of user cognition and improving AI personalization.

Contribution

It is the first dataset to include user thoughts in real-world multi-turn conversations, facilitating research on cognitive dynamics in human-AI interactions.

Findings

Thoughts are semantically distinct from messages and hard for LLMs to infer.

Thoughts improve user-behavior prediction and aid in personalized assistant training.

ThoughtTrace captures diverse, long-horizon interactions across multiple language models.

Abstract

Conversational AI has now reached billions of users, yet existing datasets capture only what people say, not what they think. We introduce ThoughtTrace, the first large-scale dataset that pairs real-world multi-turn human--AI conversations with users' self-reported thoughts: their reasons for sending prompts and reactions to assistant responses. ThoughtTrace comprises 1,058 users, 2,155 conversations, 17,058 turns, and 10,174 thought annotations collected across 20 language models. Our analysis shows that ThoughtTrace captures long-horizon, topically diverse interactions, and that thoughts are semantically distinct from messages, difficult for frontier LLMs to infer from context, diverse in content, and tied to conversation stages. We further demonstrate the utility of thoughts for downstream modeling. First, thoughts improve user-behavior prediction as inference-time context. Second, thought-guided rewrites provide fine-grained alignment signals for training personalized assistants. Together, ThoughtTrace establishes user thoughts as a new data modality for studying the cognitive dynamics behind human--AI interaction and provides a foundation for building assistants that better understand and adapt to users' latent goals, preferences, and needs.