Joint angle model based learning to refine kinematic human pose estimation

TL;DR

This paper introduces a joint angle-based modeling approach to improve marker-free human pose estimation by refining keypoint accuracy and trajectories using a novel neural network trained on high-quality, angle-approximated datasets.

Contribution

It proposes a robust joint angle-based model and a bidirectional recurrent network for pose refinement, overcoming dataset inaccuracies in existing methods.

Findings

Outperforms state-of-the-art HPE refinement networks in challenging scenarios

Effectively corrects wrongly recognized joints and smooths trajectories

Demonstrates significant improvements in figure skating and breaking cases

Abstract

Marker-free human pose estimation (HPE) has found increasing applications in various fields. Current HPE suffers from occasional errors in keypoint recognition and random fluctuation in keypoint trajectories when analyzing kinematic human poses. The performance of existing deep learning-based models for HPE refinement is considerably limited by inaccurate training datasets in which the keypoints are manually annotated. This paper proposed a novel method to overcome the difficulty through joint angle-based modeling. The key techniques include: (i) A joint angle-based model of human pose, which is robust to describe kinematic human poses; (ii) Approximating temporal variation of joint angles through high order Fourier series to get reliable "ground truth"; (iii) A bidirectional recurrent network is designed as a post-processing module to refine the estimation of well-established HRNet. Trained with the high-quality dataset constructed using our method, the network demonstrates outstanding performance to correct wrongly recognized joints and smooth their spatiotemporal trajectories. Tests show that joint angle-based refinement (JAR) outperforms the state-of-the-art HPE refinement network in challenging cases like figure skating and breaking.