Multipath Routing for Multi-Hop UAV Networks

TL;DR

This paper introduces a novel multipath routing method for multi-hop UAV networks that dynamically splits traffic to meet latency requirements, using a decentralized deep reinforcement learning approach.

Contribution

It proposes a traffic-adaptive multipath routing technique with a new MADRL algorithm, IPPO-DM, for optimizing traffic splitting ratios in UAV networks.

Findings

IPPO-DM outperforms benchmark schemes in latency and packet loss.

The Dirichlet-based policy ensures feasible traffic splitting ratios.

Simulation results validate the effectiveness of the proposed method.

Abstract

Multi-hop uncrewed aerial vehicle (UAV) networks are promising to extend the terrestrial network coverage. Existing multi-hop UAV networks employ a single routing path by selecting the next-hop forwarding node in a hop-by-hop manner, which leads to local congestion and increases traffic delays. In this paper, a novel traffic-adaptive multipath routing method is proposed for multi-hop UAV networks, which enables each UAV to dynamically split and forward traffic flows across multiple next-hop neighbors, thus meeting latency requirements of diverse traffic flows in dynamic mobile environments. An on-time packet delivery ratio maximization problem is formulated to determine the traffic splitting ratios at each hop. This sequential decision-making problem is modeled as a decentralized partially observable Markov decision process (Dec-POMDP). To solve this Dec-POMDP, a novel multi-agent deep reinforcement leaning (MADRL) algorithm, termed Independent Proximal Policy Optimization with Dirichlet Modeling (IPPO-DM), is developed. Specifically, the IPPO serves as the core optimization framework, where the Dirichlet distribution is leveraged to parameterize a continuous stochastic policy network on the probability simplex, inherently ensuring feasible traffic splitting ratios. Simulation results demonstrate that IPPO-DM outperforms benchmark schemes in terms of both delivery latency guarantee and packet loss performance.