Hierarchical Multi-Agent Reinforcement Learning with Control Barrier Functions for Safety-Critical Autonomous Systems

TL;DR

This paper introduces a hierarchical multi-agent reinforcement learning framework utilizing control barrier functions to ensure safety in autonomous systems, significantly enhancing safety and performance in complex multi-agent navigation tasks.

Contribution

It presents a novel hierarchical RL approach with CBFs, decomposing safety and cooperation into two learning levels for improved safety guarantees.

Findings

Achieves near-perfect safety success rate within 5% margin.

Outperforms existing methods in safety and efficiency.

Successfully navigates complex multi-agent environments.

Abstract

We address the problem of safe policy learning in multi-agent safety-critical autonomous systems. In such systems, it is necessary for each agent to meet the safety requirements at all times while also cooperating with other agents to accomplish the task. Toward this end, we propose a safe Hierarchical Multi-Agent Reinforcement Learning (HMARL) approach based on Control Barrier Functions (CBFs). Our proposed hierarchical approach decomposes the overall reinforcement learning problem into two levels learning joint cooperative behavior at the higher level and learning safe individual behavior at the lower or agent level conditioned on the high-level policy. Specifically, we propose a skill-based HMARL-CBF algorithm in which the higher level problem involves learning a joint policy over the skills for all the agents and the lower-level problem involves learning policies to execute the skills safely with CBFs. We validate our approach on challenging environment scenarios whereby a large number of agents have to safely navigate through conflicting road networks. Compared with existing state of the art methods, our approach significantly improves the safety achieving near perfect (within 5%) success/safety rate while also improving performance across all the environments.