Shared Control with Black Box Agents using Oracle Queries

TL;DR

This paper introduces a framework for shared control in robotics where agents can query an oracle for guidance, improving learning efficiency and control policy accuracy through strategic query heuristics.

Contribution

It extends shared control to include oracle queries with different response types and proposes three heuristics for optimal query timing to enhance learning.

Findings

Querying improves control policy learning.

Heuristics effectively balance query costs and benefits.

Empirical results demonstrate the advantages of oracle-based shared control.

Abstract

Shared control problems involve a robot learning to collaborate with a human. When learning a shared control policy, short communication between the agents can often significantly reduce running times and improve the system's accuracy. We extend the shared control problem to include the ability to directly query a cooperating agent. We consider two types of potential responses to a query, namely oracles: one that can provide the learner with the best action they should take, even when that action might be myopically wrong, and one with a bounded knowledge limited to its part of the system. Given this additional information channel, this work further presents three heuristics for choosing when to query: reinforcement learning-based, utility-based, and entropy-based. These heuristics aim to reduce a system's overall learning cost. Empirical results on two environments show the benefits of querying to learn a better control policy and the tradeoffs between the proposed heuristics.