Unsupervised View-Invariant Human Posture Representation

TL;DR

This paper introduces an unsupervised method for extracting view-invariant 3D human pose representations from 2D images, eliminating the need for 3D skeleton data and improving cross-view action recognition.

Contribution

The novel approach learns view-invariant human pose features from 2D images without 3D data, leveraging view and equivariance properties, and demonstrates superior performance on multiple datasets.

Findings

Improves cross-view action classification accuracy on NTU RGB+D.

Achieves first unsupervised cross-view and cross-subject rank correlation on QMAR.

Marginally outperforms supervised methods on QMAR.

Abstract

Most recent view-invariant action recognition and performance assessment approaches rely on a large amount of annotated 3D skeleton data to extract view-invariant features. However, acquiring 3D skeleton data can be cumbersome, if not impractical, in in-the-wild scenarios. To overcome this problem, we present a novel unsupervised approach that learns to extract view-invariant 3D human pose representation from a 2D image without using 3D joint data. Our model is trained by exploiting the intrinsic view-invariant properties of human pose between simultaneous frames from different viewpoints and their equivariant properties between augmented frames from the same viewpoint. We evaluate the learned view-invariant pose representations for two downstream tasks. We perform comparative experiments that show improvements on the state-of-the-art unsupervised cross-view action classification accuracy on NTU RGB+D by a significant margin, on both RGB and depth images. We also show the efficiency of transferring the learned representations from NTU RGB+D to obtain the first ever unsupervised cross-view and cross-subject rank correlation results on the multi-view human movement quality dataset, QMAR, and marginally improve on the-state-of-the-art supervised results for this dataset. We also carry out ablation studies to examine the contributions of the different components of our proposed network.