The Center of Attention: Center-Keypoint Grouping via Attention for Multi-Person Pose Estimation

TL;DR

CenterGroup introduces an attention-based, end-to-end trainable framework for multi-person pose estimation that outperforms existing bottom-up methods in accuracy and speed by leveraging transformer-based grouping of keypoints.

Contribution

The paper presents a novel attention mechanism for grouping keypoints into person centers, enabling fully differentiable and faster multi-person pose estimation.

Findings

Achieves state-of-the-art accuracy in multi-person pose estimation.

Increases inference speed by up to 2.5 times compared to previous bottom-up methods.

Provides an end-to-end trainable framework with improved grouping accuracy.

Abstract

We introduce CenterGroup, an attention-based framework to estimate human poses from a set of identity-agnostic keypoints and person center predictions in an image. Our approach uses a transformer to obtain context-aware embeddings for all detected keypoints and centers and then applies multi-head attention to directly group joints into their corresponding person centers. While most bottom-up methods rely on non-learnable clustering at inference, CenterGroup uses a fully differentiable attention mechanism that we train end-to-end together with our keypoint detector. As a result, our method obtains state-of-the-art performance with up to 2.5x faster inference time than competing bottom-up methods. Our code is available at https://github.com/dvl-tum/center-group .