The OPS-SAT benchmark for detecting anomalies in satellite telemetry

TL;DR

This paper introduces OPSSAT-AD, a publicly available satellite telemetry dataset with ground-truth annotations, enabling the development and validation of automated anomaly detection algorithms for space missions.

Contribution

The paper provides the first publicly available, annotated satellite telemetry dataset from OPS-SAT and baseline results for various anomaly detection algorithms, facilitating reproducible research.

Findings

Baseline results for 30 anomaly detection algorithms are provided.

OPSSAT-AD dataset enables fair comparison of anomaly detection methods.

A set of quality metrics for evaluating anomaly detection algorithms is proposed.

Abstract

Detecting anomalous events in satellite telemetry is a critical task in space operations. This task, however, is extremely time-consuming, error-prone and human dependent, thus automated data-driven anomaly detection algorithms have been emerging at a steady pace. However, there are no publicly available datasets of real satellite telemetry accompanied with the ground-truth annotations that could be used to train and verify anomaly detection supervised models. In this article, we address this research gap and introduce the AI-ready benchmark dataset (OPSSAT-AD) containing the telemetry data acquired on board OPS-SAT -- a CubeSat mission which has been operated by the European Space Agency which has come to an end during the night of 22--23 May 2024 (CEST). The dataset is accompanied with the baseline results obtained using 30 supervised and unsupervised classic and deep machine learning algorithms for anomaly detection. They were trained and validated using the training-test dataset split introduced in this work, and we present a suggested set of quality metrics which should be always calculated to confront the new algorithms for anomaly detection while exploiting OPSSAT-AD. We believe that this work may become an important step toward building a fair, reproducible and objective validation procedure that can be used to quantify the capabilities of the emerging anomaly detection techniques in an unbiased and fully transparent way.