Explicablility as Minimizing Distance from Expected Behavior

TL;DR

This paper proposes a method to improve human-AI collaboration by modeling plan explicability as the distance from expected behavior, using learned heuristics to generate more understandable plans.

Contribution

It introduces a regression model for plan explicability based on plan distance and an anytime search algorithm to produce progressively explicable plans.

Findings

Effective in simulated autonomous car domain

Improves plan explicability scores

Applicable to physical robot domain

Abstract

In order to have effective human-AI collaboration, it is necessary to address how the AI agent's behavior is being perceived by the humans-in-the-loop. When the agent's task plans are generated without such considerations, they may often demonstrate inexplicable behavior from the human's point of view. This problem may arise due to the human's partial or inaccurate understanding of the agent's planning model. This may have serious implications from increased cognitive load to more serious concerns of safety around a physical agent. In this paper, we address this issue by modeling plan explicability as a function of the distance between a plan that agent makes and the plan that human expects it to make. We learn a regression model for mapping the plan distances to explicability scores of plans and develop an anytime search algorithm that can use this model as a heuristic to come up with progressively explicable plans. We evaluate the effectiveness of our approach in a simulated autonomous car domain and a physical robot domain.