Exploiting Local Observations for Robust Robot Learning

TL;DR

This paper demonstrates that multi-agent reinforcement learning with local observations can match centralized control performance while offering enhanced robustness to failures and disturbances in robotic systems.

Contribution

It provides a theoretical and empirical comparison between centralized and decentralized MARL, emphasizing the importance of observability for robustness in robotic learning.

Findings

Decentralized MARL achieves comparable performance to centralized methods.

MARL with local observations exhibits greater resilience to perturbations.

Real-world experiments confirm improved robustness of decentralized controllers.

Abstract

While many robotic tasks can be addressed using either centralized single-agent control with full state observation or decentralized multi-agent control, clear criteria for choosing between these approaches remain underexplored. This paper systematically investigates how multi-agent reinforcement learning (MARL) with local observations can improve robustness in complex robotic systems compared to traditional centralized control. Through theoretical analysis and empirical validation, we show that in certain tasks, decentralized MARL can achieve performance comparable to centralized methods while exhibiting greater resilience to perturbations and agent failures. By analytically demonstrating the equivalence of single-agent reinforcement learning (SARL) and MARL under full observability, we identify observability as the critical factor distinguishing the two paradigms. We further derive bounds quantifying performance degradation under external perturbations for locally observable policies. Empirical results on standard MARL benchmarks confirm that MARL with limited observations can maintain competitive performance. Finally, real-world experiments with a mobile manipulator demonstrate that decentralized MARL controllers achieve markedly improved robustness to agent malfunctions and environmental disturbances relative to centralized baselines. Together, these findings highlight MARL with local observations as a robust and practical alternative to conventional centralized control in complex robotic systems.