Dessie: Disentanglement for Articulated 3D Horse Shape and Pose Estimation from Images

TL;DR

Dessie is a novel method that uses synthetic data generation and disentanglement techniques to accurately estimate 3D shape and pose of horses from images, overcoming data limitations and generalizing to other large animals.

Contribution

Introducing the first approach that combines synthetic data and disentanglement for 3D animal shape and pose estimation, specifically focusing on horses.

Findings

Outperforms existing 3D horse reconstruction methods.

Generalizes effectively to other large animals like zebras, cows, and deer.

Uses synthetic data with text-based texture generation for training.

Abstract

In recent years, 3D parametric animal models have been developed to aid in estimating 3D shape and pose from images and video. While progress has been made for humans, it's more challenging for animals due to limited annotated data. To address this, we introduce the first method using synthetic data generation and disentanglement to learn to regress 3D shape and pose. Focusing on horses, we use text-based texture generation and a synthetic data pipeline to create varied shapes, poses, and appearances, learning disentangled spaces. Our method, Dessie, surpasses existing 3D horse reconstruction methods and generalizes to other large animals like zebras, cows, and deer. See the project website at: \url{https://celiali.github.io/Dessie/}.