How do you Converse with an Analytical Chatbot? Revisiting Gricean Maxims for Designing Analytical Conversational Behavior

TL;DR

This paper investigates how Gricean Maxims can guide the design of analytical chatbots, exploring user expectations and system behaviors through Wizard of Oz studies and analyzing different interface variants for effective data-driven conversations.

Contribution

It introduces a framework based on Gricean Maxims for designing analytical chatbots and provides empirical insights into user preferences and system performance across different interface types.

Findings

Preferences for intent interpretation vary by interface

Voice + chart interface enhances data understanding

Trust is influenced by system response appropriateness

Abstract

Chatbots have garnered interest as conversational interfaces for a variety of tasks. While general design guidelines exist for chatbot interfaces, little work explores analytical chatbots that support conversing with data. We explore Gricean Maxims to help inform the basic design of effective conversational interaction. We also draw inspiration from natural language interfaces for data exploration to support ambiguity and intent handling. We ran Wizard of Oz studies with 30 participants to evaluate user expectations for text and voice chatbot design variants. Results identified preferences for intent interpretation and revealed variations in user expectations based on the interface affordances. We subsequently conducted an exploratory analysis of three analytical chatbot systems (text + chart, voice + chart, voice-only) that implement these preferred design variants. Empirical evidence from a second 30-participant study informs implications specific to data-driven conversation such as interpreting intent, data orientation, and establishing trust through appropriate system responses.