On Multi-Agent Deep Deterministic Policy Gradients and their Explainability for SMARTS Environment

TL;DR

This paper compares two multi-agent deep reinforcement learning algorithms, MAPPO and MADDPG, in the SMARTS autonomous driving simulator, analyzing their performance and explainability for cooperative multi-agent scenarios.

Contribution

It introduces a comparative analysis of MAPPO and MADDPG in SMARTS, highlighting their strengths, weaknesses, and explainability aspects in autonomous driving tasks.

Findings

MAPPO outperforms MADDPG in cooperative scenarios

Explainability of RL approaches improves with waypoint integration

Identifies potential improvements for multi-agent RL in autonomous driving

Abstract

Multi-Agent RL or MARL is one of the complex problems in Autonomous Driving literature that hampers the release of fully-autonomous vehicles today. Several simulators have been in iteration after their inception to mitigate the problem of complex scenarios with multiple agents in Autonomous Driving. One such simulator--SMARTS, discusses the importance of cooperative multi-agent learning. For this problem, we discuss two approaches--MAPPO and MADDPG, which are based on-policy and off-policy RL approaches. We compare our results with the state-of-the-art results for this challenge and discuss the potential areas of improvement while discussing the explainability of these approaches in conjunction with waypoints in the SMARTS environment.