Deep Reinforcement Learning for Real-Time Ground Delay Program Revision and Corresponding Flight Delay Assignments

TL;DR

This study applies reinforcement learning models to optimize Ground Delay Programs in air traffic management, aiming to improve efficiency amid uncertainties, but faces challenges in effective learning due to environmental simplifications.

Contribution

Introduces RL-based models for real-time GDP revision using realistic simulation environments and complex reward functions, highlighting the potential and challenges of RL in ATM.

Findings

Models struggled to learn effectively in the simulated environment.

Environmental assumptions may have oversimplified real-world complexities.

The paper provides insights into challenges of applying RL in ATM.

Abstract

This paper explores the optimization of Ground Delay Programs (GDP), a prevalent Traffic Management Initiative used in Air Traffic Management (ATM) to reconcile capacity and demand discrepancies at airports. Employing Reinforcement Learning (RL) to manage the inherent uncertainties in the national airspace system-such as weather variability, fluctuating flight demands, and airport arrival rates-we developed two RL models: Behavioral Cloning (BC) and Conservative Q-Learning (CQL). These models are designed to enhance GDP efficiency by utilizing a sophisticated reward function that integrates ground and airborne delays and terminal area congestion. We constructed a simulated single-airport environment, SAGDP_ENV, which incorporates real operational data along with predicted uncertainties to facilitate realistic decision-making scenarios. Utilizing the whole year 2019 data from Newark Liberty International Airport (EWR), our models aimed to preemptively set airport program rates. Despite thorough modeling and simulation, initial outcomes indicated that the models struggled to learn effectively, attributed potentially to oversimplified environmental assumptions. This paper discusses the challenges encountered, evaluates the models' performance against actual operational data, and outlines future directions to refine RL applications in ATM.