Scalable Operator Allocation for Multi-Robot Assistance: A Restless Bandit Approach

TL;DR

This paper introduces a scalable method for allocating human operators to multiple semi-autonomous robots using a restless bandit approach, enabling efficient decision-making in complex, large-scale systems.

Contribution

It derives verifiable conditions for indexability in multi-robot operator allocation, facilitating the application of the Whittle index heuristic for scalable solutions.

Findings

Whittle index policy outperforms existing methods in simulations.

Conditions for indexability hold for various robot transition models.

The approach is scalable to systems with many robots and operators.

Abstract

In this paper, we consider the problem of allocating human operators in a system with multiple semi-autonomous robots. Each robot is required to perform an independent sequence of tasks, subjected to a chance of failing and getting stuck in a fault state at every task. If and when required, a human operator can assist or teleoperate a robot. Conventional MDP techniques used to solve such problems face scalability issues due to exponential growth of state and action spaces with the number of robots and operators. In this paper we derive conditions under which the operator allocation problem is indexable, enabling the use of the Whittle index heuristic. The conditions can be easily checked to verify indexability, and we show that they hold for a wide range of problems of interest. Our key insight is to leverage the structure of the value function of individual robots, resulting in conditions that can be verified separately for each state of each robot. We apply these conditions to two types of transitions commonly seen in remote robot supervision systems. Through numerical simulations, we demonstrate the efficacy of Whittle index policy as a near-optimal and scalable approach that outperforms existing scalable methods.