Dynamic Competency Self-Assessment for Autonomous Agents

TL;DR

This paper introduces ET-GOA, an online algorithm enabling autonomous agents to self-assess their task competency dynamically, improving human understanding of robot capabilities in uncertain environments.

Contribution

The paper presents ET-GOA, a novel event-triggered statistical method for real-time competency assessment in autonomous agents, advancing self-assessment capabilities.

Findings

ET-GOA effectively generates competency reports during tasks.

The algorithm responds adaptively to environmental uncertainties.

Experimental results demonstrate improved self-awareness in autonomous agents.

Abstract

As autonomous robots are deployed in increasingly complex environments, platform degradation, environmental uncertainties, and deviations from validated operation conditions can make it difficult for human partners to understand robot capabilities and limitations. The ability for a robot to self-assess its competency in dynamic and uncertain environments will be a crucial next step in successful human-robot teaming. This work presents and evaluates an Event-Triggered Generalized Outcome Assessment (ET-GOA) algorithm for autonomous agents to dynamically assess task confidence during execution. The algorithm uses a fast online statistical test of the agent's observations and its model predictions to decide when competency assessment is needed. We provide experimental results using ET-GOA to generate competency reports during a simulated delivery task and suggest future research directions for self-assessing agents.