React to This! How Humans Challenge Interactive Agents using Nonverbal Behaviors

TL;DR

This study analyzes how humans use nonverbal behaviors to challenge and test the capabilities of various interactive agents, providing a detailed codebook to inform the design of more believable robots.

Contribution

It introduces a comprehensive codebook of 188 nonverbal behaviors used by humans to challenge different types of agents, advancing understanding of human-agent interactions.

Findings

Humans employ diverse nonverbal behaviors to test agents physically, emotionally, and socially.

Participants interacted with a variety of agents, including humanoid robots and non-humanoid objects.

Insights from behaviors can inform the development of more interaction-aware and believable robots.

Abstract

How do people use their faces and bodies to test the interactive abilities of a robot? Making lively, believable agents is often seen as a goal for robots and virtual agents but believability can easily break down. In this Wizard-of-Oz (WoZ) study, we observed 1169 nonverbal interactions between 20 participants and 6 types of agents. We collected the nonverbal behaviors participants used to challenge the characters physically, emotionally, and socially. The participants interacted freely with humanoid and non-humanoid forms: a robot, a human, a penguin, a pufferfish, a banana, and a toilet. We present a human behavior codebook of 188 unique nonverbal behaviors used by humans to test the virtual characters. The insights and design strategies drawn from video observations aim to help build more interaction-aware and believable robots, especially when humans push them to their limits.