Action-Agnostic Human Pose Forecasting

TL;DR

This paper introduces an action-agnostic hierarchical RNN model called TP-RNN for accurate short- and long-term human pose forecasting without activity labels, outperforming existing methods on benchmark datasets.

Contribution

The paper presents a novel hierarchical RNN architecture that models multi-scale temporal dependencies in human pose sequences without relying on activity labels.

Findings

Outperforms state-of-the-art methods quantitatively.

Effective in both short- and long-term pose forecasting.

Validated on Human 3.6M and Penn Action datasets.

Abstract

Predicting and forecasting human dynamics is a very interesting but challenging task with several prospective applications in robotics, health-care, etc. Recently, several methods have been developed for human pose forecasting; however, they often introduce a number of limitations in their settings. For instance, previous work either focused only on short-term or long-term predictions, while sacrificing one or the other. Furthermore, they included the activity labels as part of the training process, and require them at testing time. These limitations confine the usage of pose forecasting models for real-world applications, as often there are no activity-related annotations for testing scenarios. In this paper, we propose a new action-agnostic method for short- and long-term human pose forecasting. To this end, we propose a new recurrent neural network for modeling the hierarchical and multi-scale characteristics of the human dynamics, denoted by triangular-prism RNN (TP-RNN). Our model captures the latent hierarchical structure embedded in temporal human pose sequences by encoding the temporal dependencies with different time-scales. For evaluation, we run an extensive set of experiments on Human 3.6M and Penn Action datasets and show that our method outperforms baseline and state-of-the-art methods quantitatively and qualitatively. Codes are available at https://github.com/eddyhkchiu/pose_forecast_wacv/