Adversarial Generative Grammars for Human Activity Prediction

TL;DR

This paper introduces an adversarial generative grammar model that captures temporal dependencies to predict multiple plausible future human activities and poses, outperforming existing methods on several datasets.

Contribution

It presents a novel adversarial grammar framework that learns stochastic production rules and latent representations for diverse future activity prediction.

Findings

Outperforms state-of-the-art methods in activity prediction accuracy.

Effectively models multiple plausible future outcomes.

Excels in predicting further into the future.

Abstract

In this paper we propose an adversarial generative grammar model for future prediction. The objective is to learn a model that explicitly captures temporal dependencies, providing a capability to forecast multiple, distinct future activities. Our adversarial grammar is designed so that it can learn stochastic production rules from the data distribution, jointly with its latent non-terminal representations. Being able to select multiple production rules during inference leads to different predicted outcomes, thus efficiently modeling many plausible futures. The adversarial generative grammar is evaluated on the Charades, MultiTHUMOS, Human3.6M, and 50 Salads datasets and on two activity prediction tasks: future 3D human pose prediction and future activity prediction. The proposed adversarial grammar outperforms the state-of-the-art approaches, being able to predict much more accurately and further in the future, than prior work.