Anomaly Detection in Video Data Based on Probabilistic Latent Space Models

TL;DR

This paper introduces a novel anomaly detection method in video data using Variational Autoencoders for dimensionality reduction and a Markov Jump Particle Filter for prediction, tailored for autonomous vehicle applications.

Contribution

It combines VAE-based latent space modeling with a specialized particle filter to detect anomalies in video sequences for autonomous vehicles, a novel integration for this purpose.

Findings

Effective in detecting anomalies in various vehicle scenarios

Able to predict future frames with high accuracy

Suitable for multi-modal autonomous vehicle systems

Abstract

This paper proposes a method for detecting anomalies in video data. A Variational Autoencoder (VAE) is used for reducing the dimensionality of video frames, generating latent space information that is comparable to low-dimensional sensory data (e.g., positioning, steering angle), making feasible the development of a consistent multi-modal architecture for autonomous vehicles. An Adapted Markov Jump Particle Filter defined by discrete and continuous inference levels is employed to predict the following frames and detecting anomalies in new video sequences. Our method is evaluated on different video scenarios where a semi-autonomous vehicle performs a set of tasks in a closed environment.