The LICORS Cabinet: Nonparametric Algorithms for Spatio-temporal Prediction

TL;DR

This paper introduces three simple nonparametric algorithms based on light cone decompositions for efficient and probabilistic prediction of high-dimensional spatio-temporal data like videos, exploiting local structure.

Contribution

The paper presents novel light cone algorithms that enable tractable, assumption-light, and probabilistic inference for complex spatio-temporal prediction tasks.

Findings

Algorithms achieve good predictive performance.

Methods enable distributional predictions over spatio-temporal data.

Approach applicable to high-dimensional data like full-frame videos.

Abstract

Spatio-temporal data is intrinsically high dimensional, so unsupervised modeling is only feasible if we can exploit structure in the process. When the dynamics are local in both space and time, this structure can be exploited by splitting the global field into many lower-dimensional "light cones". We review light cone decompositions for predictive state reconstruction, introducing three simple light cone algorithms. These methods allow for tractable inference of spatio-temporal data, such as full-frame video. The algorithms make few assumptions on the underlying process yet have good predictive performance and can provide distributions over spatio-temporal data, enabling sophisticated probabilistic inference.