Equivariant Multi-agent Reinforcement Learning for Multimodal Vehicle-to-Infrastructure Systems

TL;DR

This paper introduces an equivariant multi-agent reinforcement learning framework for multimodal vehicle-to-infrastructure systems, leveraging symmetries and graph neural networks to improve policy coordination and network performance.

Contribution

It proposes a novel self-supervised learning approach with equivariant GNN policies for decentralized V2I systems, enhancing accuracy and efficiency over existing methods.

Findings

Achieved over two-fold accuracy improvements with multimodal sensing.

Attained more than 50% performance gains over standard MARL approaches.

Demonstrated the effectiveness of equivariant policies in a simulated V2I environment.

Abstract

In this paper, we study a vehicle-to-infrastructure (V2I) system where distributed base stations (BSs) acting as road-side units (RSUs) collect multimodal (wireless and visual) data from moving vehicles. We consider a decentralized rate maximization problem, where each RSU relies on its local observations to optimize its resources, while all RSUs must collaborate to guarantee favorable network performance. We recast this problem as a distributed multi-agent reinforcement learning (MARL) problem, by incorporating rotation symmetries in terms of vehicles' locations. To exploit these symmetries, we propose a novel self-supervised learning framework where each BS agent aligns the latent features of its multimodal observation to extract the positions of the vehicles in its local region. Equipped with this sensing data at each RSU, we train an equivariant policy network using a graph neural network (GNN) with message passing layers, such that each agent computes its policy locally, while all agents coordinate their policies via a signaling scheme that overcomes partial observability and guarantees the equivariance of the global policy. We present numerical results carried out in a simulation environment, where ray-tracing and computer graphics are used to collect wireless and visual data. Results show the generalizability of our self-supervised and multimodal sensing approach, achieving more than two-fold accuracy gains over baselines, and the efficiency of our equivariant MARL training, attaining more than 50% performance gains over standard approaches.