Resolving Congestions in the Air Traffic Management Domain via Multiagent Reinforcement Learning Methods

TL;DR

This paper explores the use of multiagent reinforcement learning to optimize flight delays for resolving air traffic congestion, demonstrating promising results in real-world scenarios and offering insights into solution quality.

Contribution

It introduces a novel MARL framework formalized as a MA-MDP and Markov game for pre-tactical air traffic delay management, addressing dynamic, multiagent decision-making challenges.

Findings

Effective resolution of demand-capacity problems demonstrated

MARL methods outperform traditional approaches in case studies

Visualizations aid in understanding solution quality

Abstract

In this article, we report on the efficiency and effectiveness of multiagent reinforcement learning methods (MARL) for the computation of flight delays to resolve congestion problems in the Air Traffic Management (ATM) domain. Specifically, we aim to resolve cases where demand of airspace use exceeds capacity (demand-capacity problems), via imposing ground delays to flights at the pre-tactical stage of operations (i.e. few days to few hours before operation). Casting this into the multiagent domain, agents, representing flights, need to decide on own delays w.r.t. own preferences, having no information about others' payoffs, preferences and constraints, while they plan to execute their trajectories jointly with others, adhering to operational constraints. Specifically, we formalize the problem as a multiagent Markov Decision Process (MA-MDP) and we show that it can be considered as a Markov game in which interacting agents need to reach an equilibrium: What makes the problem more interesting is the dynamic setting in which agents operate, which is also due to the unforeseen, emergent effects of their decisions in the whole system. We propose collaborative multiagent reinforcement learning methods to resolve demand-capacity imbalances: Extensive experimental study on real-world cases, shows the potential of the proposed approaches in resolving problems, while advanced visualizations provide detailed views towards understanding the quality of solutions provided.