Predictive Learning: Using Future Representation Learning Variantial Autoencoder for Human Action Prediction

TL;DR

This paper introduces a novel predictive learning framework using a variant of a variational autoencoder to generate future frames, enhancing human action prediction accuracy by focusing on future scenario features.

Contribution

The paper proposes a new variational autoencoder-based model that generates future frames, improving human action prediction by emphasizing future-related features.

Findings

Future frame generation improves prediction accuracy.

The model outperforms existing methods on UT and UCF101 datasets.

Higher scores achieved with partial observation compared to traditional methods.

Abstract

The unsupervised Pretraining method has been widely used in aiding human action recognition. However, existing methods focus on reconstructing the already present frames rather than generating frames which happen in future.In this paper, We propose an improved Variantial Autoencoder model to extract the features with a high connection to the coming scenarios, also known as Predictive Learning. Our framework lists as following: two steam 3D-convolution neural networks are used to extract both spatial and temporal information as latent variables. Then a resample method is introduced to create new normal distribution probabilistic latent variables and finally, the deconvolution neural network will use these latent variables generate next frames. Through this possess, we train the model to focus more on how to generate the future and thus it will extract the future high connected features. In the experiment stage, A large number of experiments on UT and UCF101 datasets reveal that future generation aids Prediction does improve the performance. Moreover, the Future Representation Learning Network reach a higher score than other methods when in half observation. This means that Future Representation Learning is better than the traditional Representation Learning and other state- of-the-art methods in solving the human action prediction problems to some extends.