Pose estimator and tracker using temporal flow maps for limbs

TL;DR

This paper introduces temporal flow maps for limbs (TML) and a multi-stride approach to improve human pose estimation and tracking in videos by effectively utilizing temporal information between frames.

Contribution

It proposes a novel TML representation and a multi-stride data augmentation method for end-to-end pose estimation and tracking in videos.

Findings

Efficient pose estimation and tracking on PoseTrack datasets.

Improved accuracy with temporal flow maps and multi-stride augmentation.

Effective learning of spatial and temporal features in a unified network.

Abstract

For human pose estimation in videos, it is significant how to use temporal information between frames. In this paper, we propose temporal flow maps for limbs (TML) and a multi-stride method to estimate and track human poses. The proposed temporal flow maps are unit vectors describing the limbs' movements. We constructed a network to learn both spatial information and temporal information end-to-end. Spatial information such as joint heatmaps and part affinity fields is regressed in the spatial network part, and the TML is regressed in the temporal network part. We also propose a data augmentation method to learn various types of TML better. The proposed multi-stride method expands the data by randomly selecting two frames within a defined range. We demonstrate that the proposed method efficiently estimates and tracks human poses on the PoseTrack 2017 and 2018 datasets.