Learning What Information to Give in Partially Observed Domains

TL;DR

This paper develops algorithms for autonomous agents to decide what information to transmit to humans in partially observed environments, optimizing communication to improve human understanding and collaboration.

Contribution

It introduces a belief MDP framework and an approximate solution for planning information transmission, along with an online learning algorithm for human preferences.

Findings

Effective in simulated search-and-recover tasks

Improves human-agent collaboration

Demonstrates learning of human preferences online

Abstract

In many robotic applications, an autonomous agent must act within and explore a partially observed environment that is unobserved by its human teammate. We consider such a setting in which the agent can, while acting, transmit declarative information to the human that helps them understand aspects of this unseen environment. In this work, we address the algorithmic question of how the agent should plan out what actions to take and what information to transmit. Naturally, one would expect the human to have preferences, which we model information-theoretically by scoring transmitted information based on the change it induces in weighted entropy of the human's belief state. We formulate this setting as a belief MDP and give a tractable algorithm for solving it approximately. Then, we give an algorithm that allows the agent to learn the human's preferences online, through exploration. We validate our approach experimentally in simulated discrete and continuous partially observed search-and-recover domains. Visit http://tinyurl.com/chitnis-corl-18 for a supplementary video.