RAMOTS: A Real-Time System for Aerial Multi-Object Tracking based on Deep Learning and Big Data Technology

TL;DR

This paper introduces RAMOTS, a real-time multi-object tracking system for UAV videos that combines deep learning models with big data technologies like Apache Kafka and Spark, achieving high accuracy and speed.

Contribution

The paper presents a novel integrated framework that combines scalable big data tools with state-of-the-art deep learning models for real-time aerial multi-object tracking.

Findings

Achieves a HOTA of 48.14 and MOTA of 43.51 on Visdrone2019-MOT.

Maintains 28 FPS processing speed on a single GPU.

Demonstrates effective integration of big data tech with deep learning for UAV tracking.

Abstract

Multi-object tracking (MOT) in UAV-based video is challenging due to variations in viewpoint, low resolution, and the presence of small objects. While other research on MOT dedicated to aerial videos primarily focuses on the academic aspect by developing sophisticated algorithms, there is a lack of attention to the practical aspect of these systems. In this paper, we propose a novel real-time MOT framework that integrates Apache Kafka and Apache Spark for efficient and fault-tolerant video stream processing, along with state-of-the-art deep learning models YOLOv8/YOLOv10 and BYTETRACK/BoTSORT for accurate object detection and tracking. Our work highlights the importance of not only the advanced algorithms but also the integration of these methods with scalable and distributed systems. By leveraging these technologies, our system achieves a HOTA of 48.14 and a MOTA of 43.51 on the Visdrone2019-MOT test set while maintaining a real-time processing speed of 28 FPS on a single GPU. Our work demonstrates the potential of big data technologies and deep learning for addressing the challenges of MOT in UAV applications.