SCAPE: A Simple and Strong Category-Agnostic Pose Estimator

TL;DR

SCAPE introduces a simplified, attention-based approach for category-agnostic pose estimation, achieving superior accuracy and efficiency over prior methods by focusing on feature matching within a streamlined architecture.

Contribution

The paper proposes a simple, strong baseline for CAPE using pure self-attention and introduces two modules to enhance attention quality, outperforming prior arts in accuracy and speed.

Findings

Outperforms prior methods by 2.2 and 1.3 PCK in 1-shot and 5-shot settings

Faster inference speed and lighter model capacity

Effective attention process with global keypoint features and keypoint attention refiner

Abstract

Category-Agnostic Pose Estimation (CAPE) aims to localize keypoints on an object of any category given few exemplars in an in-context manner. Prior arts involve sophisticated designs, e.g., sundry modules for similarity calculation and a two-stage framework, or takes in extra heatmap generation and supervision. We notice that CAPE is essentially a task about feature matching, which can be solved within the attention process. Therefore we first streamline the architecture into a simple baseline consisting of several pure self-attention layers and an MLP regression head -- this simplification means that one only needs to consider the attention quality to boost the performance of CAPE. Towards an effective attention process for CAPE, we further introduce two key modules: i) a global keypoint feature perceptor to inject global semantic information into support keypoints, and ii) a keypoint attention refiner to enhance inter-node correlation between keypoints. They jointly form a Simple and strong Category-Agnostic Pose Estimator (SCAPE). Experimental results show that SCAPE outperforms prior arts by 2.2 and 1.3 PCK under 1-shot and 5-shot settings with faster inference speed and lighter model capacity, excelling in both accuracy and efficiency. Code and models are available at https://github.com/tiny-smart/SCAPE