Evaluating ML Robustness in GNSS Interference Classification, Characterization & Localization

TL;DR

This paper assesses the robustness of machine learning models in classifying, characterizing, and localizing GNSS jamming devices using a new extensive dataset, analyzing uncertainties and environmental effects to improve real-world applicability.

Contribution

It introduces a large-scale dataset for GNSS interference analysis and evaluates the performance and uncertainty of 129 vision encoder models across multiple tasks.

Findings

ML models show resilience to environmental variations

Certain models outperform others in localization accuracy

Uncertainty analysis aids in understanding model reliability

Abstract

Jamming devices disrupt signals from the global navigation satellite system (GNSS) and pose a significant threat, as they compromise the robustness of accurate positioning. The detection of anomalies within frequency snapshots is crucial to counteract these interferences effectively. A critical preliminary countermeasure involves the reliable classification of interferences and the characterization and localization of jamming devices. This paper introduces an extensive dataset comprising snapshots obtained from a low-frequency antenna that capture various generated interferences within a large-scale environment, including controlled multipath effects. Our objective is to assess the resilience of machine learning (ML) models against environmental changes, such as multipath effects, variations in interference attributes, such as interference class, bandwidth, and signal power, the accuracy of jamming device localization, and the constraints imposed by snapshot input lengths. Furthermore, we evaluate the performance of a diverse set of 129 distinct vision encoder models across all tasks. By analyzing the aleatoric and epistemic uncertainties, we demonstrate the adaptability of our model in generalizing across diverse facets, thus establishing its suitability for real-world applications. Dataset: https://gitlab.cc-asp.fraunhofer.de/darcy_gnss/controlled_low_frequency