Multi-Robot Multi-Queue Control via Exhaustive Assignment Actor-Critic Learning

TL;DR

This paper introduces an adaptive actor-critic learning approach for real-time multi-robot task allocation in multi-queue systems, outperforming traditional rules especially under asymmetric conditions.

Contribution

It develops a novel exhaustive-assignment actor-critic policy that learns efficient robot-queue allocations, adapting to asymmetry and outperforming existing methods.

Findings

The proposed policy reduces discounted holding costs compared to ESL baseline.

It achieves smaller mean queue lengths across various scenarios.

The method remains near-optimal where benchmarks are available.

Abstract

We study online task allocation for multi-robot, multi-queue systems with asymmetric stochastic arrivals and switching delays. We formulate the problem in discrete time: each location can host at most one robot per slot, servicing a task consumes one slot, switching between locations incurs a one-slot travel delay, and arrivals at locations are independent Bernoulli processes with heterogeneous rates. Building on our previous structural result that optimal policies are of exhaustive type, we formulate a discounted-cost Markov decision process and develop an exhaustive-assignment actor-critic policy architecture that enforces exhaustive service by construction and learns only the next-queue allocation for idle robots. Unlike the exhaustive-serve-longest (ESL) queue rule, whose optimality is known only under symmetry, the proposed policy adapts to asymmetry in arrival rates. Across different server-location ratios, loads, and asymmetric arrival profiles, the proposed policy consistently achieves lower discounted holding cost and smaller mean queue length than the ESL baseline, while remaining near-optimal on instances where an optimal benchmark is available. These results show that structure-aware actor-critic methods provide an effective approach for real-time multi-robot scheduling.