BTranspose: Bottleneck Transformers for Human Pose Estimation with Self-Supervised Pre-Training

TL;DR

This paper introduces BTranspose, a model combining Bottleneck Transformers with self-supervised pre-training for improved 2D human pose estimation, achieving competitive accuracy with fewer parameters.

Contribution

It integrates Bottleneck Transformers with self-supervised pre-training for the first time in human pose estimation, enhancing accuracy and interpretability.

Findings

Achieves 76.4 AP on COCO validation set.

Pre-training with DINO improves prediction accuracy.

Provides insights into keypoint dependencies on transformer layers.

Abstract

The task of 2D human pose estimation is challenging as the number of keypoints is typically large (~ 17) and this necessitates the use of robust neural network architectures and training pipelines that can capture the relevant features from the input image. These features are then aggregated to make accurate heatmap predictions from which the final keypoints of human body parts can be inferred. Many papers in literature use CNN-based architectures for the backbone, and/or combine it with a transformer, after which the features are aggregated to make the final keypoint predictions [1]. In this paper, we consider the recently proposed Bottleneck Transformers [2], which combine CNN and multi-head self attention (MHSA) layers effectively, and we integrate it with a Transformer encoder and apply it to the task of 2D human pose estimation. We consider different backbone architectures and pre-train them using the DINO self-supervised learning method [3], this pre-training is found to improve the overall prediction accuracy. We call our model BTranspose, and experiments show that on the COCO validation set, our model achieves an AP of 76.4, which is competitive with other methods such as [1] and has fewer network parameters. Furthermore, we also present the dependencies of the final predicted keypoints on both the MHSA block and the Transformer encoder layers, providing clues on the image sub-regions the network attends to at the mid and high levels.