Investigating Low Data, Confidence Aware Image Prediction on Smooth Repetitive Videos using Gaussian Processes

TL;DR

This paper introduces a data-efficient, confidence-aware image prediction method using Gaussian Processes, capable of predicting future images with interpretable uncertainty bounds from limited data, applicable to real-world scenarios.

Contribution

The paper presents a novel non-parametric, Gaussian Process-based approach for future image prediction that provides confidence metrics and works effectively with small datasets.

Findings

Accurately predicts future images with confidence bounds from limited data.

Demonstrates effectiveness on fluid simulations, pedestrian flows, and weather patterns.

Provides interpretable uncertainty estimates alongside predictions.

Abstract

The ability to predict future states is crucial to informed decision-making while interacting with dynamic environments. With cameras providing a prevalent and information-rich sensing modality, the problem of predicting future states from image sequences has garnered a lot of attention. Current state-of-the-art methods typically train large parametric models for their predictions. Though often able to predict with accuracy these models often fail to provide interpretable confidence metrics around their predictions. Additionally these methods are reliant on the availability of large training datasets to converge to useful solutions. In this paper, we focus on the problem of predicting future images of an image sequence with interpretable confidence bounds from very little training data. To approach this problem, we use non-parametric models to take a probabilistic approach to image prediction. We generate probability distributions over sequentially predicted images, and propagate uncertainty through time to generate a confidence metric for our predictions. Gaussian Processes are used for their data efficiency and ability to readily incorporate new training data online. Our methods predictions are evaluated on a smooth fluid simulation environment. We showcase the capabilities of our approach on real world data by predicting pedestrian flows and weather patterns from satellite imagery.