Reframing Conversational Design in HRI: Deliberate Design with AI Scaffolds

TL;DR

This paper introduces ACE, a system that leverages AI and LLMs to support deliberate, structured design of human-robot conversations, improving prompt clarity and interaction quality.

Contribution

The paper presents a novel AI-Aided Conversation Engine (ACE) that scaffolds prompt creation, collects grounded feedback, and refines prompts using LLMs, addressing inefficiencies in HRI conversational design.

Findings

ACE enables clearer, more specific robot prompts

Prompts created with ACE lead to higher-quality interactions

User studies confirm ACE's effectiveness in design and interaction quality

Abstract

Large language models (LLMs) have enabled conversational robots to move beyond constrained dialogue toward free-form interaction. However, without context-specific adaptation, generic LLM outputs can be ineffective or inappropriate. This adaptation is often attempted through prompt engineering, which is non-intuitive and tedious. Moreover, predominant design practice in HRI relies on impression-based, trial-and-error refinement without structured methods or tools, making the process inefficient and inconsistent. To address this, we present the AI-Aided Conversation Engine (ACE), a system that supports the deliberate design of human-robot conversations. ACE contributes three key innovations: 1) an LLM-powered voice agent that scaffolds initial prompt creation to overcome the "blank page problem," 2) an annotation interface that enables the collection of granular and grounded feedback on conversational transcripts, and 3) using LLMs to translate user feedback into prompt refinements. We evaluated ACE through two user studies, examining both designs' experience and end users' interactions with robots designed using ACE. Results show that ACE facilitates the creation of robot behavior prompts with greater clarity and specificity, and that the prompts generated with ACE lead to higher-quality human-robot conversational interactions.