Uncertainty-Aware Anticipation of Activities

TL;DR

This paper introduces a probabilistic approach for long-term activity prediction in videos, effectively capturing multiple possible future activities and their durations, thus addressing uncertainty in extended time horizons.

Contribution

It presents a novel uncertainty-aware model that predicts diverse future activity sequences by modeling probability distributions, improving long-term activity anticipation.

Findings

Successfully captures multi-modal future activities

Maintains accuracy in single-sequence predictions

Performs well on challenging datasets

Abstract

Anticipating future activities in video is a task with many practical applications. While earlier approaches are limited to just a few seconds in the future, the prediction time horizon has just recently been extended to several minutes in the future. However, as increasing the predicted time horizon, the future becomes more uncertain and models that generate a single prediction fail at capturing the different possible future activities. In this paper, we address the uncertainty modelling for predicting long-term future activities. Both an action model and a length model are trained to model the probability distribution of the future activities. At test time, we sample from the predicted distributions multiple samples that correspond to the different possible sequences of future activities. Our model is evaluated on two challenging datasets and shows a good performance in capturing the multi-modal future activities without compromising the accuracy when predicting a single sequence of future activities.