Improving Competence for Reliable Autonomy

TL;DR

This paper presents a method for semi-autonomous systems to improve their reliability by learning from human feedback, enhancing their competence and decision-making over time in complex, unstructured environments.

Contribution

It introduces a technique for competence-aware systems to identify and incorporate missing features into their models through online feedback, improving reliability.

Findings

Enhanced prediction of human involvement

Improved system competence over deployment

Better performance in complex environments

Abstract

Given the complexity of real-world, unstructured domains, it is often impossible or impractical to design models that include every feature needed to handle all possible scenarios that an autonomous system may encounter. For an autonomous system to be reliable in such domains, it should have the ability to improve its competence online. In this paper, we propose a method for improving the competence of a system over the course of its deployment. We specifically focus on a class of semi-autonomous systems known as competence-aware systems that model their own competence -- the optimal extent of autonomy to use in any given situation -- and learn this competence over time from feedback received through interactions with a human authority. Our method exploits such feedback to identify important state features missing from the system's initial model, and incorporates them into its state representation. The result is an agent that better predicts human involvement, leading to improvements in its competence and reliability, and as a result, its overall performance.