Multi-Agent Meta-Advisor for UAV Fleet Trajectory Design in Vehicular Networks

TL;DR

This paper introduces a multi-agent meta-advisor framework for UAV fleet trajectory planning in vehicular networks, enabling adaptive, efficient deployment of aerial base stations to enhance urban connectivity.

Contribution

It proposes a novel multi-agent meta-advisor with override mechanism that improves exploration and convergence in decentralized deep reinforcement learning for UAV trajectory optimization.

Findings

MAMO outperforms epsilon-greedy in convergence speed and rewards.

UABS deployment significantly improves network coverage and performance.

Framework adapts effectively across diverse urban scenarios.

Abstract

Future vehicular networks require continuous connectivity to serve highly mobile users in urban environments. To mitigate the coverage limitations of fixed terrestrial macro base stations (MBS) under non line-of-sight (NLoS) conditions, fleets of unmanned aerial base stations (UABSs) can be deployed as aerial base stations, dynamically repositioning to track vehicular users and traffic hotspots in coordination with the terrestrial network. This paper addresses cooperative multi-agent trajectory design under different service areas and takeoff configurations, where rapid and safe adaptation across scenarios is essential. We formulate the problem as a multi-task decentralized partially observable Markov decision process and solve it using centralized training and decentralized execution with double dueling deep Q-network (3DQN), enabling online training for real-world deployments. However, efficient exploration remains a bottleneck, with conventional strategies like $\epsilon$-greedy requiring careful tuning. To overcome this, we propose the multi-agent meta-advisor with advisor override (MAMO). This framework guides agent exploration through a meta-policy learned jointly across tasks. It uses a dynamic override mechanism that allows agents to reject misaligned guidance when the advisor fails to generalize to a specific scenario. Simulation results across three realistic urban scenarios and multiple takeoff configurations show that MAMO achieves faster convergence and higher returns than tuned $\epsilon$-greedy baselines, outperforming both an advisor-only ablation and a single generalized policy. Finally, we demonstrate that the learned UABS fleet significantly improves network performance compared to deployments without aerial support.