A Horse with no Labels: Self-Supervised Horse Pose Estimation from Unlabelled Images and Synthetic Prior

TL;DR

This paper introduces a fully self-supervised approach for horse pose estimation that requires only unlabelled images and a small set of synthetic 2D poses, eliminating the need for extensive annotations or complex models.

Contribution

It presents a novel self-supervised method that learns accurate 3D and 2D animal poses using minimal assumptions and synthetic data, without relying on pose annotations.

Findings

Accurately estimates 3D and 2D horse poses from unlabelled images.

Requires only a small synthetic dataset of 2D poses, three times smaller than the image dataset.

Demonstrates robustness on challenging horse images with minimal prior assumptions.

Abstract

Obtaining labelled data to train deep learning methods for estimating animal pose is challenging. Recently, synthetic data has been widely used for pose estimation tasks, but most methods still rely on supervised learning paradigms utilising synthetic images and labels. Can training be fully unsupervised? Is a tiny synthetic dataset sufficient? What are the minimum assumptions that we could make for estimating animal pose? Our proposal addresses these questions through a simple yet effective self-supervised method that only assumes the availability of unlabelled images and a small set of synthetic 2D poses. We completely remove the need for any 3D or 2D pose annotations (or complex 3D animal models), and surprisingly our approach can still learn accurate 3D and 2D poses simultaneously. We train our method with unlabelled images of horses mainly collected for YouTube videos and a prior consisting of 2D synthetic poses. The latter is three times smaller than the number of images needed for training. We test our method on a challenging set of horse images and evaluate the predicted 3D and 2D poses. We demonstrate that it is possible to learn accurate animal poses even with as few assumptions as unlabelled images and a small set of 2D poses generated from synthetic data. Given the minimum requirements and the abundance of unlabelled data, our method could be easily deployed to different animals.