Relational Q-Functionals: Multi-Agent Learning to Recover from Unforeseen Robot Malfunctions in Continuous Action Domains

TL;DR

The paper introduces Relational Q-Functionals (RQF), a framework that enhances multi-agent learning adaptability and resilience in continuous domains, enabling robots to recover from unforeseen malfunctions by leveraging relational networks.

Contribution

The paper presents RQF, a novel relational network-based approach that improves multi-agent learning adaptability and malfunction recovery in continuous action spaces.

Findings

RQF enables effective cooperation among agents.

RQF allows robots to recover from unexpected malfunctions.

Empirical results demonstrate improved resilience in robotic modules.

Abstract

Cooperative multi-agent learning methods are essential in developing effective cooperation strategies in multi-agent domains. In robotics, these methods extend beyond multi-robot scenarios to single-robot systems, where they enable coordination among different robot modules (e.g., robot legs or joints). However, current methods often struggle to quickly adapt to unforeseen failures, such as a malfunctioning robot leg, especially after the algorithm has converged to a strategy. To overcome this, we introduce the Relational Q-Functionals (RQF) framework. RQF leverages a relational network, representing agents' relationships, to enhance adaptability, providing resilience against malfunction(s). Our algorithm also efficiently handles continuous state-action domains, making it adept for robotic learning tasks. Our empirical results show that RQF enables agents to use these relationships effectively to facilitate cooperation and recover from an unexpected malfunction in single-robot systems with multiple interacting modules. Thus, our approach offers promising applications in multi-agent systems, particularly in scenarios with unforeseen malfunctions.