Steady-State Strategy Synthesis for Swarms of Autonomous Agents

TL;DR

This paper explores the challenge of synthesizing steady-state policies for multiagent systems, revealing computational hardness and proposing an efficient algorithm for a specific subclass, validated through experiments.

Contribution

It analyzes the complexity of steady-state synthesis for multiagent systems and introduces a scalable algorithm for a subclass of memoryless profiles.

Findings

Computational hardness results for multiagent steady-state synthesis.

Memoryless profiles are insufficient for approximating achievable frequency vectors.

An efficient algorithm improves performance over naive strategy sharing methods.

Abstract

Steady-state synthesis aims to construct a policy for a given MDP $D$ such that the long-run average frequencies of visits to the vertices of $D$ satisfy given numerical constraints. This problem is solvable in polynomial time, and memoryless policies are sufficient for approximating an arbitrary frequency vector achievable by a general (infinite-memory) policy. We study the steady-state synthesis problem for multiagent systems, where multiple autonomous agents jointly strive to achieve a suitable frequency vector. We show that the problem for multiple agents is computationally hard (PSPACE or NP hard, depending on the variant), and memoryless strategy profiles are insufficient for approximating achievable frequency vectors. Furthermore, we prove that even evaluating the frequency vector achieved by a given memoryless profile is computationally hard. This reveals a severe barrier to constructing an efficient synthesis algorithm, even for memoryless profiles. Nevertheless, we design an efficient and scalable synthesis algorithm for a subclass of full memoryless profiles, and we evaluate this algorithm on a large class of randomly generated instances. The experimental results demonstrate a significant improvement against a naive algorithm based on strategy sharing.