Fault Tolerant Multi-Agent Learning with Adversarial Budget Constraints
David Mguni, Yaqi Sun, Haojun Chen, Wanrong Yang, Amir Darabi, Larry Olanrewaju Orimoloye, Yaodong Yang
TL;DR
This paper introduces MARTA, a robustness layer for multi-agent reinforcement learning that enhances fault tolerance by modeling agent malfunctions as a Markov game, with proven convergence and significant empirical improvements.
Contribution
MARTA is a novel plug-and-play robustness mechanism that integrates with existing MARL algorithms, providing theoretical guarantees and practical robustness against agent malfunctions.
Findings
MARTA achieves up to 116.7% performance improvement in SMAC.
It significantly reduces failure rates under fault regimes.
The Bellman operator in MARTA is a contraction, ensuring convergence.
Abstract
We study robustness to agent malfunctions in cooperative multi-agent reinforcement learning (MARL), a failure mode that is critical in practice yet underexplored in existing theory. We introduce MARTA, a plug-and-play robustness layer that augments standard MARL algorithms with a Switcher-Adversary mechanism which selectively induces malfunctions in performance-critical states. This formulation defines a fault-switching -player Markov game in which the Switcher chooses when and which agent fails, and the Adversary controls the resulting faulty behaviour via random or worst-case policies. We develop a Q-learning-type scheme and show that the associated Bellman operator is a contraction, yielding existence and uniqueness of the minimax value, convergence to a Markov perfect equilibrium. MARTA integrates seamlessly with MARL algorithms without architectural modification and…
Peer Reviews
Decision·Submitted to ICLR 2026
- The exact problem setting tackled seems to be novel, to the best of my knowledge, although there are other works (though uncited) that tackle very similar problems - Experimental results suggest the proposed framework improves robustness compared to without applying the proposed framework
- Related works [A, B, C] tackling very similar attack vectors as the one proposed in this paper are not cited or discussed. - The experimental section is a bit lacking. In particular, the proposed method is not compared to any other MARL methods that try to improve fault tolerance, including both methods discussed in the paper (e.g., M3DDPG) as well as ones that the authors did not identify but are highly relevant (e.g., [A]). - The domains that are tested on seem quite simple - Also, details o
1. The paper addresses an important and relatively unexplored problem of fault tolerance in multi-agent reinforcement learning, with a clear and well-motivated problem statement. 2. The proposed MARTA framework introduces a novel Switcher–Adversary formulation that models both the timing and behavior of agent malfunctions in a unified way. 3. The work offers theoretical guarantees and consistent empirical results across various MARLs, demonstrating clear practical benefits and robustness impro
1. The presentation is quite dry and mathematically heavy, making the paper difficult to follow in parts. The figures are limited and do not clearly illustrate the key intuitions behind the proposed framework. 2. The title and abstract do not accurately convey the main emphasis of the work. They make the paper appear as a system or framework for fault-tolerant MARL, whereas the actual contribution lies more in the theoretical formulation and convergence analysis of the proposed approach. 3.
The paper addresses an important (yet well-studied) problem in cooperative MARL. The proposed method, MARTA, and setting are built on well-established foundations, such as zero-sum Markov games, and are therefore sound. I appreciate the diversity of environments used to demonstrate the effectiveness of MARTA.
**Novelty** The problem of faulty agents or where each agent has an adversarial counterpart has been addressed in prior work, which has neither been discussed in related work nor compared with in the experiments [1,2]. The problem setting defined in Section 2.1 has been formalized as a zero-sum Markov game or, more generally, as a mixed cooperative-competitive game in the literature [3,4]. The adversarial training scheme of two opposing QMIX instances has also been introduced in [1]. This lea
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsReinforcement Learning in Robotics · Adversarial Robustness in Machine Learning · Game Theory and Applications