Real-time processing of high-resolution video and 3D model-based tracking for remote towers

TL;DR

This paper presents a system for real-time processing and 3D tracking of high-resolution video data in remote tower operations, enhancing decision-making with efficient, resilient software architecture.

Contribution

It introduces a method to efficiently process high-res video and track objects in 3D in real-time on a single workstation using decoupled processes and parallel hardware.

Findings

Achieved real-time processing of 25 million pixels at 30 fps.

Implemented decoupled processes for improved resilience.

Enhanced ground movement tracking accuracy.

Abstract

High quality video data is a core component in emerging remote tower operations as it inherently contains a huge amount of information on which an air traffic controller can base decisions. Various digital technologies also have the potential to exploit this data to bring enhancements, including tracking ground movements by relating events in the video view to their positions in 3D space. The total resolution of remote tower setups with multiple cameras often exceeds 25 million RGB pixels and is captured at 30 frames per second or more. It is thus a challenge to efficiently process all the data in such a way as to provide relevant real-time enhancements to the controller. In this paper we discuss how a number of improvements can be implemented efficiently on a single workstation by decoupling processes and utilizing hardware for parallel computing. We also highlight how decoupling the processes in this way increases resilience of the software solution in the sense that failure of a single component does not impair the function of the other components.