Human operator cognitive availability aware Mixed-Initiative control

TL;DR

This paper introduces a cognitive availability aware mixed-initiative control system for remote robots, enabling dynamic autonomy level switching based on operator cognitive state, improving disaster response operations.

Contribution

It presents a novel MI controller that infers operator cognitive availability using computer vision to inform autonomy switching, advancing previous MI systems.

Findings

Effective assistance in disaster response exploration tasks

Improved operator workload management

Insights into evaluating complex MI controllers

Abstract

This paper presents a Cognitive Availability Aware Mixed-Initiative Controller for remotely operated mobile robots. The controller enables dynamic switching between different levels of autonomy (LOA), initiated by either the AI or the human operator. The controller leverages a state-of-the-art computer vision method and an off-the-shelf web camera to infer the cognitive availability of the operator and inform the AI-initiated LOA switching. This constitutes a qualitative advancement over previous Mixed-Initiative (MI) controllers. The controller is evaluated in a disaster response experiment, in which human operators have to conduct an exploration task with a remote robot. MI systems are shown to effectively assist the operators, as demonstrated by quantitative and qualitative results in performance and workload. Additionally, some insights into the experimental difficulties of evaluating complex MI controllers are presented.