One-shot Policy Elicitation via Semantic Reward Manipulation

TL;DR

This paper introduces SPEAR, a novel algorithm that uses semantic explanations to modify agents' reward functions, enabling effective one-shot policy manipulation and improved collaboration in complex domains.

Contribution

SPEAR is a new sequential optimization method that leverages semantic explanations to efficiently manipulate policies, outperforming existing approaches in runtime and problem complexity.

Findings

SPEAR effectively manipulates policies in complex domains.

It significantly reduces runtime compared to state-of-the-art methods.

The method enhances agent collaboration through semantic reward augmentation.

Abstract

Synchronizing expectations and knowledge about the state of the world is an essential capability for effective collaboration. For robots to effectively collaborate with humans and other autonomous agents, it is critical that they be able to generate intelligible explanations to reconcile differences between their understanding of the world and that of their collaborators. In this work we present Single-shot Policy Explanation for Augmenting Rewards (SPEAR), a novel sequential optimization algorithm that uses semantic explanations derived from combinations of planning predicates to augment agents' reward functions, driving their policies to exhibit more optimal behavior. We provide an experimental validation of our algorithm's policy manipulation capabilities in two practically grounded applications and conclude with a performance analysis of SPEAR on domains of increasingly complex state space and predicate counts. We demonstrate that our method makes substantial improvements over the state-of-the-art in terms of runtime and addressable problem size, enabling an agent to leverage its own expertise to communicate actionable information to improve another's performance.