Of Mice and Pose: 2D Mouse Pose Estimation from Unlabelled Data and Synthetic Prior

TL;DR

This paper introduces a novel self-supervised approach for estimating 2D mouse poses from unlabelled images by leveraging synthetic data and a pose prior, enabling animal behavior analysis without manual annotations.

Contribution

It adapts a human pose estimation method to mice using synthetic 3D models to generate a pose prior, eliminating the need for manual annotation in animal pose estimation.

Findings

Promising pose estimation results on mouse videos without paired training data.

Comparable performance to supervised methods in animal pose estimation.

Potential to extend the approach to other animal species like horses.

Abstract

Numerous fields, such as ecology, biology, and neuroscience, use animal recordings to track and measure animal behaviour. Over time, a significant volume of such data has been produced, but some computer vision techniques cannot explore it due to the lack of annotations. To address this, we propose an approach for estimating 2D mouse body pose from unlabelled images using a synthetically generated empirical pose prior. Our proposal is based on a recent self-supervised method for estimating 2D human pose that uses single images and a set of unpaired typical 2D poses within a GAN framework. We adapt this method to the limb structure of the mouse and generate the empirical prior of 2D poses from a synthetic 3D mouse model, thereby avoiding manual annotation. In experiments on a new mouse video dataset, we evaluate the performance of the approach by comparing pose predictions to a manually obtained ground truth. We also compare predictions with those from a supervised state-of-the-art method for animal pose estimation. The latter evaluation indicates promising results despite the lack of paired training data. Finally, qualitative results using a dataset of horse images show the potential of the setting to adapt to other animal species.