FCPose: Fully Convolutional Multi-Person Pose Estimation with Dynamic Instance-Aware Convolutions

TL;DR

FCPose introduces a fully convolutional, dynamic instance-aware approach to multi-person pose estimation that eliminates ROI operations, enabling faster inference with comparable or better accuracy than existing methods.

Contribution

It presents a novel fully convolutional framework with dynamic keypoint heads that are conditioned on each instance, improving speed and simplicity over prior ROI-based methods.

Findings

Achieves 4.5x faster inference than Mask R-CNN on COCO

Maintains almost constant inference time regardless of person count

Outperforms state-of-the-art methods in speed/accuracy trade-off

Abstract

We propose a fully convolutional multi-person pose estimation framework using dynamic instance-aware convolutions, termed FCPose. Different from existing methods, which often require ROI (Region of Interest) operations and/or grouping post-processing, FCPose eliminates the ROIs and grouping post-processing with dynamic instance-aware keypoint estimation heads. The dynamic keypoint heads are conditioned on each instance (person), and can encode the instance concept in the dynamically-generated weights of their filters. Moreover, with the strong representation capacity of dynamic convolutions, the keypoint heads in FCPose are designed to be very compact, resulting in fast inference and making FCPose have almost constant inference time regardless of the number of persons in the image. For example, on the COCO dataset, a real-time version of FCPose using the DLA-34 backbone infers about 4.5x faster than Mask R-CNN (ResNet-101) (41.67 FPS vs. 9.26FPS) while achieving improved performance. FCPose also offers better speed/accuracy trade-off than other state-of-the-art methods. Our experiment results show that FCPose is a simple yet effective multi-person pose estimation framework. Code is available at: https://git.io/AdelaiDet