Talk Me Through It: Developing Effective Systems for Chart Authoring
Nazar Ponochevnyi, Young-Ho Kim, Joseph Jay Williams, Anastasia Kuzminykh

TL;DR
This paper investigates the differences in command structures between spoken and typed chart creation instructions and demonstrates that training systems on spoken imagined-chart data significantly improves their performance.
Contribution
It provides empirical analysis of instruction differences and shows the benefits of training on spoken imagined-chart data for better chart-authoring systems.
Findings
Spoken imagined-chart instructions have richer command structures.
Systems trained on spoken data outperform those trained on typed data.
Design guidelines are proposed for real-world chart-authoring systems.
Abstract
Recent chart-authoring systems increasingly focus on natural-language input, enabling users to form a mental image of the chart they wish to create and express this intent using spoken instructions (spoken imagined-chart data). Yet these systems are predominantly trained on typed instructions written while viewing the target chart (typed existing-chart data). While the cognitive processes for describing an existing chart arguably differ from those for creating a new chart, the structural differences in the corresponding prompts remain underexplored. We present empirical findings on the structural differences among spoken imagined-chart instructions, typed imagined-chart instructions, and typed existing-chart instructions for chart creation, showing that imagined-chart prompts contain richer command formats, element specifications, and complex linguistic features, especially in spoken…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsSpeech and dialogue systems · Interactive and Immersive Displays · Visual and Cognitive Learning Processes
