Adaptive Neighborhood Resizing for Stochastic Reachability in Multi-Agent Systems

TL;DR

DAMPC is a novel distributed algorithm that adaptively resizes neighborhoods and horizons in multi-agent systems to efficiently solve stochastic reachability problems, maintaining convergence with improved speed.

Contribution

Introduces DAMPC, an adaptive, distributed control algorithm that dynamically adjusts neighborhood size and horizon for efficient stochastic reachability in multi-agent systems.

Findings

DAMPC doubles speed compared to non-adaptive methods.

Maintains convergence with smaller neighborhoods and horizons.

Effective for any controllable multi-agent system with a cost function.

Abstract

We present DAMPC, a distributed, adaptive-horizon and adaptive-neighborhood algorithm for solving the stochastic reachability problem in multi-agent systems, in particular flocking modeled as a Markov decision process. At each time step, every agent calls a centralized, adaptive-horizon model-predictive control (AMPC) algorithm to obtain an optimal solution for its local neighborhood. Second, the agents derive the flock-wide optimal solution through a sequence of consensus rounds. Third, the neighborhood is adaptively resized using a flock-wide, cost-based Lyapunov function V. This way DAMPC improves efficiency without compromising convergence. We evaluate DAMPC's performance using statistical model checking. Our results demonstrate that, compared to AMPC, DAMPC achieves considerable speed-up (two-fold in some cases) with only a slightly lower rate of convergence. The smaller average neighborhood size and lookahead horizon demonstrate the benefits of the DAMPC approach for stochastic reachability problems involving any controllable multi-agent system that possesses a cost function.