Reducing Human-Robot Goal State Divergence with Environment Design

TL;DR

This paper introduces a new metric called Goal State Divergence (GSD) to quantify and reduce the mismatch between human and robot goal states through environment design modifications, improving human-robot collaboration.

Contribution

The paper proposes GSD as a novel metric for goal alignment and a method to identify minimal environment changes to prevent goal state mismatches.

Findings

GSD effectively measures goal state divergence in human-robot interactions.

Environment modifications based on GSD reduce goal mismatches in benchmarks.

The approach improves safety and alignment in human-robot collaboration.

Abstract

One of the most difficult challenges in creating successful human-AI collaborations is aligning a robot's behavior with a human user's expectations. When this fails to occur, a robot may misinterpret their specified goals, prompting it to perform actions with unanticipated, potentially dangerous side effects. To avoid this, we propose a new metric we call Goal State Divergence $\mathcal{(GSD)}$, which represents the difference between a robot's final goal state and the one a human user expected. In cases where $\mathcal{GSD}$ cannot be directly calculated, we show how it can be approximated using maximal and minimal bounds. We then input the $\mathcal{GSD}$ value into our novel human-robot goal alignment (HRGA) design problem, which identifies a minimal set of environment modifications that can prevent mismatches like this. To show the effectiveness of $\mathcal{GSD}$ for reducing differences between human-robot goal states, we empirically evaluate our approach on several standard benchmarks.