Sequential keypoint density estimator: an overlooked baseline of skeleton-based video anomaly detection

TL;DR

SeeKer is a simple yet effective skeleton-based anomaly detection method that models keypoint density sequences autoregressively, outperforming previous approaches on multiple datasets in safety-critical video analysis.

Contribution

It introduces a novel autoregressive density estimation approach at the keypoint level for skeleton sequences, providing a strong baseline for anomaly detection.

Findings

Outperforms previous methods on UBnormal and MSAD-HR datasets

Achieves competitive results on ShanghaiTech dataset

Utilizes confidence-weighted log-conditionals for anomaly scoring

Abstract

Detecting anomalous human behaviour is an important visual task in safety-critical applications such as healthcare monitoring, workplace safety, or public surveillance. In these contexts, abnormalities are often reflected with unusual human poses. Thus, we propose SeeKer, a method for detecting anomalies in sequences of human skeletons. Our method formulates the skeleton sequence density through autoregressive factorization at the keypoint level. The corresponding conditional distributions represent probable keypoint locations given prior skeletal motion. We formulate the joint distribution of the considered skeleton as causal prediction of conditional Gaussians across its constituent keypoints. A skeleton is flagged as anomalous if its keypoint locations surprise our model (i.e. receive a low density). In practice, our anomaly score is a weighted sum of per-keypoint log-conditionals, where the weights account for the confidence of the underlying keypoint detector. Despite its conceptual simplicity, SeeKer surpasses all previous methods on the UBnormal and MSAD-HR datasets while delivering competitive performance on the ShanghaiTech dataset.