Processing of Crowdsourced Observations of Aircraft in a High Performance Computing Environment

TL;DR

This paper presents a high-performance computing workflow for processing 3.9 billion crowdsourced aircraft observations from the OpenSky Network to develop models for low-altitude airspace risk assessment, with all data openly available.

Contribution

It introduces a scalable HPC workflow for processing large crowdsourced aircraft data and publicly releases the datasets used for collision risk modeling.

Findings

Processed 3.9 billion observations efficiently

Trained aircraft models with over 250,000 flight hours

All datasets are open source and publicly accessible

Abstract

As unmanned aircraft systems (UASs) continue to integrate into the U.S. National Airspace System (NAS), there is a need to quantify the risk of airborne collisions between unmanned and manned aircraft to support regulation and standards development. Both regulators and standards developing organizations have made extensive use of Monte Carlo collision risk analysis simulations using probabilistic models of aircraft flight. We've previously determined that the observations of manned aircraft by the OpenSky Network, a community network of ground-based sensors, are appropriate to develop models of the low altitude environment. This works overviews the high performance computing workflow designed and deployed on the Lincoln Laboratory Supercomputing Center to process 3.9 billion observations of aircraft. We then trained the aircraft models using more than 250,000 flight hours at 5,000 feet above ground level or below. A key feature of the workflow is that all the aircraft observations and supporting datasets are available as open source technologies or been released to the public domain.