Multi-Instance Pose Networks: Rethinking Top-Down Pose Estimation

TL;DR

This paper introduces MIPNet, a novel top-down pose estimation model that predicts multiple pose instances within a single bounding box, improving accuracy in crowded scenes with occlusions.

Contribution

MIPNet incorporates a Multi-Instance Modulation Block to adaptively predict multiple poses, addressing limitations of traditional single-instance methods in crowded environments.

Findings

Achieves 70.0 AP on CrowdPose, surpassing previous methods.

Improves 6.5 AP on OCHuman over prior art.

Maintains stable performance with fewer high-confidence boxes.

Abstract

A key assumption of top-down human pose estimation approaches is their expectation of having a single person/instance present in the input bounding box. This often leads to failures in crowded scenes with occlusions. We propose a novel solution to overcome the limitations of this fundamental assumption. Our Multi-Instance Pose Network (MIPNet) allows for predicting multiple 2D pose instances within a given bounding box. We introduce a Multi-Instance Modulation Block (MIMB) that can adaptively modulate channel-wise feature responses for each instance and is parameter efficient. We demonstrate the efficacy of our approach by evaluating on COCO, CrowdPose, and OCHuman datasets. Specifically, we achieve 70.0 AP on CrowdPose and 42.5 AP on OCHuman test sets, a significant improvement of 2.4 AP and 6.5 AP over the prior art, respectively. When using ground truth bounding boxes for inference, MIPNet achieves an improvement of 0.7 AP on COCO, 0.9 AP on CrowdPose, and 9.1 AP on OCHuman validation sets compared to HRNet. Interestingly, when fewer, high confidence bounding boxes are used, HRNet's performance degrades (by 5 AP) on OCHuman, whereas MIPNet maintains a relatively stable performance (drop of 1 AP) for the same inputs.