ZooBuilder: 2D and 3D Pose Estimation for Quadrupeds Using Synthetic Data

TL;DR

ZooBuilder presents a novel pipeline that leverages synthetic data from keyframe animations to train models for accurate 2D and 3D pose estimation of quadruped animals, facilitating wildlife animation and analysis.

Contribution

The paper introduces a new synthetic data generation method and an end-to-end pipeline for animal pose estimation, improving automation and accuracy in wildlife motion capture.

Findings

Effective training of pose estimation models using synthetic data

Successful extraction of 2D and 3D joint coordinates from wild animal videos

Potential for automating wildlife animation and behavioral studies

Abstract

This work introduces a novel strategy for generating synthetic training data for 2D and 3D pose estimation of animals using keyframe animations. With the objective to automate the process of creating animations for wildlife, we train several 2D and 3D pose estimation models with synthetic data, and put in place an end-to-end pipeline called ZooBuilder. The pipeline takes as input a video of an animal in the wild, and generates the corresponding 2D and 3D coordinates for each joint of the animal's skeleton. With this approach, we produce motion capture data that can be used to create animations for wildlife.