TEC: A Collection of Human Trial-and-error Trajectories for Problem Solving
Xinkai Zhang, Jingtao Zhan, Yiqun Liu, Qingyao Ai

TL;DR
This paper introduces TEC, a dataset capturing human trial-and-error problem-solving trajectories, to improve AI's ability to learn from human strategies in complex tasks.
Contribution
The paper presents a new data annotation platform and dataset capturing detailed human trial-and-error trajectories, filling a gap in AI training data.
Findings
Humans outperform LLMs in trial-and-error accuracy.
The dataset includes 5,370 trajectories and 41,229 webpage reflections.
TEC platform and dataset are publicly available.
Abstract
Trial-and-error is a fundamental strategy for humans to solve complex problems and a necessary capability for Artificial Intelligence (AI) systems operating in real-world environments. Although several trial-and-error AI techniques have recently been proposed, most of them rely on simple heuristics designed by researchers and achieve limited performance gains. The core issue is the absence of appropriate data: current models cannot learn from detailed records of how humans actually conduct trial-and-error in practice. To address this gap, we introduce a data annotation platform and a corresponding dataset, termed Trial-and-Error Collection (TEC). The platform records users' complete trajectories across multiple trials and collects their reflections after receiving error feedback. Using this platform, we record the problem-solving processes of 46 participants on 58 tasks, resulting in…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
