Heterogeneous Multi-task Learning for Human Pose Estimation with Deep Convolutional Neural Network

TL;DR

This paper introduces a multi-task deep learning framework for human pose estimation that jointly learns pose regression and body-part detection, improving accuracy and interpretability.

Contribution

It presents a novel heterogeneous multi-task learning approach combining pose estimation and body-part detection within a deep CNN architecture.

Findings

Achieves state-of-the-art results on multiple datasets.

Including body-part detection regularizes the network effectively.

Middle-layer neurons are tuned to specific body parts.

Abstract

We propose an heterogeneous multi-task learning framework for human pose estimation from monocular image with deep convolutional neural network. In particular, we simultaneously learn a pose-joint regressor and a sliding-window body-part detector in a deep network architecture. We show that including the body-part detection task helps to regularize the network, directing it to converge to a good solution. We report competitive and state-of-art results on several data sets. We also empirically show that the learned neurons in the middle layer of our network are tuned to localized body parts.