Learning Delicate Local Representations for Multi-Person Pose Estimation

TL;DR

This paper introduces the Residual Steps Network (RSN) and Pose Refine Machine (PRM) to improve multi-person pose estimation by capturing delicate local features and refining keypoints, achieving state-of-the-art results.

Contribution

The paper presents a novel RSN architecture for detailed local feature extraction and an attention-based PRM for better feature refinement, advancing pose estimation accuracy.

Findings

Won 1st place at COCO Keypoint Challenge 2019

Achieved 78.6 AP on COCO test-dev without extra data

Ensembled models reached 79.2 AP on COCO test-dev

Abstract

In this paper, we propose a novel method called Residual Steps Network (RSN). RSN aggregates features with the same spatial size (Intra-level features) efficiently to obtain delicate local representations, which retain rich low-level spatial information and result in precise keypoint localization. Additionally, we observe the output features contribute differently to final performance. To tackle this problem, we propose an efficient attention mechanism - Pose Refine Machine (PRM) to make a trade-off between local and global representations in output features and further refine the keypoint locations. Our approach won the 1st place of COCO Keypoint Challenge 2019 and achieves state-of-the-art results on both COCO and MPII benchmarks, without using extra training data and pretrained model. Our single model achieves 78.6 on COCO test-dev, 93.0 on MPII test dataset. Ensembled models achieve 79.2 on COCO test-dev, 77.1 on COCO test-challenge dataset. The source code is publicly available for further research at https://github.com/caiyuanhao1998/RSN/