Learning Feature Pyramids for Human Pose Estimation

TL;DR

This paper introduces a novel Pyramid Residual Module (PRMs) to improve scale invariance in deep convolutional neural networks for human pose estimation, achieving state-of-the-art results on standard benchmarks.

Contribution

We propose a new Pyramid Residual Module and extend weight initialization schemes for multi-branch networks, enhancing scale invariance and performance in pose estimation.

Findings

Achieves state-of-the-art results on pose estimation benchmarks.

Demonstrates the effectiveness of PRMs in handling scale variations.

Provides theoretical extension for weight initialization in multi-branch networks.

Abstract

Articulated human pose estimation is a fundamental yet challenging task in computer vision. The difficulty is particularly pronounced in scale variations of human body parts when camera view changes or severe foreshortening happens. Although pyramid methods are widely used to handle scale changes at inference time, learning feature pyramids in deep convolutional neural networks (DCNNs) is still not well explored. In this work, we design a Pyramid Residual Module (PRMs) to enhance the invariance in scales of DCNNs. Given input features, the PRMs learn convolutional filters on various scales of input features, which are obtained with different subsampling ratios in a multi-branch network. Moreover, we observe that it is inappropriate to adopt existing methods to initialize the weights of multi-branch networks, which achieve superior performance than plain networks in many tasks recently. Therefore, we provide theoretic derivation to extend the current weight initialization scheme to multi-branch network structures. We investigate our method on two standard benchmarks for human pose estimation. Our approach obtains state-of-the-art results on both benchmarks. Code is available at https://github.com/bearpaw/PyraNet.