A Benchmark for Unsupervised Anomaly Detection in Multi-Agent Trajectories

TL;DR

This paper introduces R-U-MAAD, a new benchmark dataset for evaluating unsupervised anomaly detection methods in multi-agent driving trajectories, addressing the lack of standardization in the field.

Contribution

It provides a novel benchmark dataset with annotated anomalies and compares multiple baseline algorithms for unsupervised anomaly detection in urban driving scenarios.

Findings

Deep auto-encoders and one-class classifiers perform well

Benchmark and models will be publicly available

11 baseline methods evaluated using standard metrics

Abstract

Human intuition allows to detect abnormal driving scenarios in situations they never experienced before. Like humans detect those abnormal situations and take countermeasures to prevent collisions, self-driving cars need anomaly detection mechanisms. However, the literature lacks a standard benchmark for the comparison of anomaly detection algorithms. We fill the gap and propose the R-U-MAAD benchmark for unsupervised anomaly detection in multi-agent trajectories. The goal is to learn a representation of the normal driving from the training sequences without labels, and afterwards detect anomalies. We use the Argoverse Motion Forecasting dataset for the training and propose a test dataset of 160 sequences with human-annotated anomalies in urban environments. To this end we combine a replay of real-world trajectories and scene-dependent abnormal driving in the simulation. In our experiments we compare 11 baselines including linear models, deep auto-encoders and one-class classification models using standard anomaly detection metrics. The deep reconstruction and end-to-end one-class methods show promising results. The benchmark and the baseline models will be publicly available.