Conveying Autonomous Robot Capabilities through Contrasting Behaviour Summaries

TL;DR

This paper introduces an adaptive search method for generating contrasting behaviour summaries of autonomous agents, supporting continuous spaces, to improve human understanding and comparison of agent capabilities.

Contribution

It presents a novel adaptive search approach for creating effective contrasting behaviour summaries in continuous state and action spaces, enhancing interpretability.

Findings

Adaptive search efficiently finds informative contrasting scenarios.

Summaries enable humans to accurately compare agent performance.

Method supports continuous state and action spaces.

Abstract

As advances in artificial intelligence enable increasingly capable learning-based autonomous agents, it becomes more challenging for human observers to efficiently construct a mental model of the agent's behaviour. In order to successfully deploy autonomous agents, humans should not only be able to understand the individual limitations of the agents but also have insight on how they compare against one another. To do so, we need effective methods for generating human interpretable agent behaviour summaries. Single agent behaviour summarization has been tackled in the past through methods that generate explanations for why an agent chose to pick a particular action at a single timestep. However, for complex tasks, a per-action explanation may not be able to convey an agents global strategy. As a result, researchers have looked towards multi-timestep summaries which can better help humans assess an agents overall capability. More recently, multi-step summaries have also been used for generating contrasting examples to evaluate multiple agents. However, past approaches have largely relied on unstructured search methods to generate summaries and require agents to have a discrete action space. In this paper we present an adaptive search method for efficiently generating contrasting behaviour summaries with support for continuous state and action spaces. We perform a user study to evaluate the effectiveness of the summaries for helping humans discern the superior autonomous agent for a given task. Our results indicate that adaptive search can efficiently identify informative contrasting scenarios that enable humans to accurately select the better performing agent with a limited observation time budget.