An Auction-based Coordination Strategy for Task-Constrained Multi-Agent Stochastic Planning with Submodular Rewards

TL;DR

This paper introduces an auction-based decentralized coordination strategy for multi-agent stochastic planning with submodular rewards, demonstrating effectiveness in drone delivery scenarios with improved scalability and near-optimal solutions.

Contribution

It presents a novel auction-based method with convergence guarantees and a neural network variant called Deep Auction for large-scale stochastic multi-agent planning.

Findings

Guarantees at least 50% optimality for the proposed method.

Deep Auction achieves scalable and efficient planning in drone delivery simulations.

Outperforms state-of-the-art methods in solution quality and efficiency.

Abstract

In many domains such as transportation and logistics, search and rescue, or cooperative surveillance, tasks are pending to be allocated with the consideration of possible execution uncertainties. Existing task coordination algorithms either ignore the stochastic process or suffer from the computational intensity. Taking advantage of the weakly coupled feature of the problem and the opportunity for coordination in advance, we propose a decentralized auction-based coordination strategy using a newly formulated score function which is generated by forming the problem into task-constrained Markov decision processes (MDPs). The proposed method guarantees convergence and at least 50% optimality in the premise of a submodular reward function. Furthermore, for the implementation on large-scale applications, an approximate variant of the proposed method, namely Deep Auction, is also suggested with the use of neural networks, which is evasive of the troublesome for constructing MDPs. Inspired by the well-known actor-critic architecture, two Transformers are used to map observations to action probabilities and cumulative rewards respectively. Finally, we demonstrate the performance of the two proposed approaches in the context of drone deliveries, where the stochastic planning for the drone league is cast into a stochastic price-collecting Vehicle Routing Problem (VRP) with time windows. Simulation results are compared with state-of-the-art methods in terms of solution quality, planning efficiency and scalability.