Deep Reinforcement Learning Aided Packet-Routing For Aeronautical Ad-Hoc Networks Formed by Passenger Planes

TL;DR

This paper introduces a deep reinforcement learning approach using DQN and DVN to optimize packet routing in aeronautical ad-hoc networks, significantly reducing end-to-end delay compared to traditional methods.

Contribution

It proposes a novel deep reinforcement learning framework with offline training and feedback mechanisms for efficient, adaptive routing in highly dynamic AANETs, outperforming benchmark protocols.

Findings

DQN-routing reduces end-to-end delay compared to benchmarks.

DVN-routing performs close to optimal routing with perfect information.

Deep learning models adapt effectively to high mobility in AANETs.

Abstract

Data packet routing in aeronautical ad-hoc networks (AANETs) is challenging due to their high-dynamic topology. In this paper, we invoke deep reinforcement learning for routing in AANETs aiming at minimizing the end-to-end (E2E) delay. Specifically, a deep Q-network (DQN) is conceived for capturing the relationship between the optimal routing decision and the local geographic information observed by the forwarding node. The DQN is trained in an offline manner based on historical flight data and then stored by each airplane for assisting their routing decisions during flight. To boost the learning efficiency and the online adaptability of the proposed DQN-routing, we further exploit the knowledge concerning the system's dynamics by using a deep value network (DVN) conceived with a feedback mechanism. Our simulation results show that both DQN-routing and DVN-routing achieve lower E2E delay than the benchmark protocol, and DVN-routing performs similarly to the optimal routing that relies on perfect global information.