PISEP^2: Pseudo Image Sequence Evolution based 3D Pose Prediction

TL;DR

This paper introduces PISEP^2, a novel framework that transforms 3D joint coordinate sequences into image sequences for efficient and accurate 3D pose prediction, suitable for real-world applications.

Contribution

The paper proposes a new skeletal representation and a non-recursive inference network for 3D pose prediction, improving efficiency and accuracy over existing methods.

Findings

Achieves state-of-the-art performance on G3D and FNTU datasets.

Models pose evolution as image sequence evolution for better correlation capture.

Reduces error accumulation with a non-recursive prediction approach.

Abstract

Pose prediction is to predict future poses given a window of previous poses. In this paper, we propose a new problem that predicts poses using 3D joint coordinate sequences. Different from the traditional pose prediction based on Mocap frames, this problem is convenient to use in real applications due to its simple sensors to capture data. We also present a new framework, PISEP^2 (Pseudo Image Sequence Evolution based 3D Pose Prediction), to address this new problem. Specifically, a skeletal representation is proposed by transforming the joint coordinate sequence into an image sequence, which can model the different correlations of different joints. With this image based skeletal representation, we model the pose prediction as the evolution of image sequence. Moreover, a novel inference network is proposed to predict all future poses in one step by decoupling the decoders in a non-recursive manner. Compared with the recursive sequence to sequence model, we can improve the computational efficiency and avoid error accumulation significantly. Extensive experiments are carried out on two benchmark datasets (e.g. G3D and FNTU). The proposed method achieves the state-of-the-art performance on both datasets, which demonstrates the effectiveness of our proposed method.