An Empirical Study on Embodied Artificial Intelligence Robot (EAIR) Software Bugs

TL;DR

This study systematically analyzes 885 bugs from EAIR projects, revealing unique symptoms and causes, especially related to AI reasoning, and provides a mapping to improve bug detection and repair in embodied AI robots.

Contribution

First comprehensive empirical analysis of EAIR system bugs, identifying 18 causes, 15 symptoms, and module associations to aid future debugging efforts.

Findings

8 EAIR-specific symptoms linked to severe failures

Major causes stem from AI reasoning and decision-making issues

Mapping of causes to modules enables targeted bug diagnosis

Abstract

Embodied Artificial Intelligence Robots (EAIR) is an emerging and rapidly evolving technological domain. Ensuring their program correctness is fundamental to their successful deployment. However, a general and in-depth understanding of EAIR system bugs remains lacking, which hinders the development of practices and techniques to tackle EAIR system bugs. To bridge this gap, we conducted the first systematic study of 885 EAIR system bugs collected from 80 EAIR system projects to investigate their symptoms, underlying causes, and module distribution. Our analysis takes considerable effort, which classifies these bugs into 18 underlying causes, 15 distinct symptoms, and identifies 13 affected modules. It reveals several new interesting findings and implications which help shed light on future research on tackling or repairing EAIR system bugs. First, among the 15 identified symptoms, our findings highlight 8 symptoms specific to EAIR systems, which is characterized by severe functional failures and potential physical hazards. Second, within the 18 underlying causes, we define 8 EAIR-specific causes, the majority of which stem from the intricate issues of AI- agent reasoning and decision making. Finally, to facilitate precise and efficient bug prediction, detection, and repair, we constructed a mapping between underlying causes and the modules in which they most frequently occur, which enables researchers to focus diagnostic efforts on the modules most susceptible to specific bug types.