# Sonic Sleight of Hand: Sound Induces Illusory Distortions in the Perception and Prediction of Robot Action

## TL;DR

This study shows that the sounds a robot makes can trick humans into thinking its movements are longer or shorter, affecting how people predict its actions.

## Contribution

The paper introduces a novel method using psychophysical tasks to measure how sound influences robot motion perception in human–robot interaction.

## Key findings

- Longer robot sounds made participants perceive movements as extending further in space.
- Sound changes also affected where participants anticipated the robot's future steps.
- The study confirms that sound design can manipulate how humans represent robot actions.

## Abstract

For efficient human–robot interaction, human operators need to be able to efficiently represent the robot’s movements in space and predict its next steps. However, according to frameworks of Bayesian multisensory integration, features outside the motion itself—like the sounds a robot makes while it moves—should affect how otherwise identical motions are perceived. Here, we translate an established psychophysical task from experimental psychology to a human–robot interaction context, which can measure these distortions to motion perception. In two series of preregistered studies, participants watched a humanoid robot make forward and backward reaching movements. When the robot hand suddenly disappeared, they reported its last seen location, either with the mouse cursor (Experiment 1a and 1b) or by matching it to probe stimuli in different locations (Experiment 2a and 2b). The results revealed that even small changes to the robot’s sound robustly affect participants’ visuospatial representation of its motions, so that the motion appeared to extend further in space when accompanied by slightly (100 ms) longer sounds compared to slightly shorter sounds (100 ms shorter). Moreover, these sound changes do not only affect where people currently locate the robot’s motion, but where they anticipate its future steps. These findings show that sound design is an effective medium for manipulating how people represent otherwise identical robot actions and coordinate its interactions with it. The study acts as proof of concept that psychophysical tasks provide a promising tool to measure how design parameters influence the perception and prediction of robot motion.

The online version contains supplementary material available at 10.1007/s12369-024-01105-5.