"Guess what I'm doing": Extending legibility to sequential decision tasks

TL;DR

This paper introduces PoL-MDP, a new method for creating legible policies in uncertain sequential decision tasks, improving interpretability and demonstrating advantages over existing approaches through simulations, user studies, and inverse reinforcement learning.

Contribution

We propose PoL-MDP, a computationally efficient approach to extend legibility to uncertain sequential decision tasks, outperforming prior deterministic and expensive methods.

Findings

PoL-MDP outperforms state-of-the-art in simulated scenarios.

Legible policies serve as better demonstrations for inverse reinforcement learning.

User study confirms improved goal inference with our legible policies.

Abstract

In this paper we investigate the notion of legibility in sequential decision tasks under uncertainty. Previous works that extend legibility to scenarios beyond robot motion either focus on deterministic settings or are computationally too expensive. Our proposed approach, dubbed PoL-MDP, is able to handle uncertainty while remaining computationally tractable. We establish the advantages of our approach against state-of-the-art approaches in several simulated scenarios of different complexity. We also showcase the use of our legible policies as demonstrations for an inverse reinforcement learning agent, establishing their superiority against the commonly used demonstrations based on the optimal policy. Finally, we assess the legibility of our computed policies through a user study where people are asked to infer the goal of a mobile robot following a legible policy by observing its actions.