Active shooter detection and robust tracking utilizing supplemental synthetic data

TL;DR

This paper presents a novel shooter detection and tracking system that leverages synthetic data, domain randomization, and transfer learning to improve robustness and enable edge hardware deployment.

Contribution

It introduces a whole-shooter detection approach using synthetic data and domain techniques, enhancing robustness and real-time edge deployment capabilities.

Findings

Effective shooter detection with synthetic data

Robust tracking despite occlusions

Real-time performance on edge devices

Abstract

The increasing concern surrounding gun violence in the United States has led to a focus on developing systems to improve public safety. One approach to developing such a system is to detect and track shooters, which would help prevent or mitigate the impact of violent incidents. In this paper, we proposed detecting shooters as a whole, rather than just guns, which would allow for improved tracking robustness, as obscuring the gun would no longer cause the system to lose sight of the threat. However, publicly available data on shooters is much more limited and challenging to create than a gun dataset alone. Therefore, we explore the use of domain randomization and transfer learning to improve the effectiveness of training with synthetic data obtained from Unreal Engine environments. This enables the model to be trained on a wider range of data, increasing its ability to generalize to different situations. Using these techniques with YOLOv8 and Deep OC-SORT, we implemented an initial version of a shooter tracking system capable of running on edge hardware, including both a Raspberry Pi and a Jetson Nano.