PointAloud: An Interaction Suite for AI-Supported Pointer-Centric Think-Aloud Computing
Frederic Gmeiner, John Thompson, George Fitzmaurice, Justin Matejka

TL;DR
PointAloud introduces AI-driven, pointer-centric interactions to enhance Think-Aloud Computing by improving user engagement, providing low-distraction feedback, and supporting rich documentation, thereby facilitating better human-AI collaboration.
Contribution
The paper presents a novel suite of pointer-centric, AI-supported interactions that address practical challenges in Think-Aloud Computing, enabling more effective verbalization and documentation.
Findings
Participants found pointer-centric interactions valuable for documentation.
AI support increased user engagement and reduced distractions.
The approach facilitated richer work process insights.
Abstract
Think-Aloud Computing, a method for capturing users' verbalized thoughts during software tasks, allows eliciting rich contextual insights into evolving intentions, struggles, and decision-making processes of users in real-time. However, existing approaches face practical challenges: users often lack awareness of what is captured by the system, are not effectively encouraged to speak, and miss or are interrupted by system feedback. Additionally, thinking aloud should feel worthwhile for users due to the gained contextual AI assistance. To better support and harness Think-Aloud Computing, we introduce PointAloud, a suite of novel AI-driven pointer-centric interactions for in-the-moment verbalization encouragement, low-distraction system feedback, and contextually rich work process documentation alongside proactive AI assistance. Our user study with 12 participants provides insights into…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsPersonal Information Management and User Behavior · Spreadsheets and End-User Computing · Usability and User Interface Design
