Orality: A Semantic Canvas for Externalizing and Clarifying Thoughts with Speech

TL;DR

Orality is a semantic visualization tool that helps users externalize, clarify, and organize spoken thoughts using interactive diagrams and AI features, outperforming speech interaction with ChatGPT in supporting thought development.

Contribution

The paper introduces Orality, a novel system combining semantic analysis and interactive visualization to enhance externalization and organization of spoken thoughts.

Findings

Orality better supports thought clarification and development than ChatGPT-based speech interaction.

Graphical and conversational tools offer complementary benefits for thinking.

Design implications for future verbalization and visualization tools are derived.

Abstract

People speak aloud to externalize thoughts as one way to help clarify and organize them. Although Speech-to-text can capture these thoughts, transcripts can be difficult to read and make sense due to disfluencies, repetitions and potential disorganization. To support thinking through verbalization, we introduce Orality, which extracts key information from spoken content, performs semantic analysis through LLMs to form a node-link diagram in an interactive canvas. Instead of reading and working with transcripts, users could manipulate clusters of nodes and give verbal instructions to re-extract and organize the content in other ways. It also provides AI-generated inspirational questions and detection of logical conflicts. We conducted a lab study with twelve participants comparing Orality against speech interaction with ChatGPT. We found that Orality can better support users in clarifying and developing their thoughts. The findings also identified the affordances of both graphical and conversational thought clarification tools and derived design implications.