SAM 3D Animal: Promptable Animal 3D Reconstruction from Images in the Wild

TL;DR

SAM 3D Animal introduces a promptable multi-animal 3D reconstruction framework from single images, leveraging a new dataset and achieving state-of-the-art results in wild animal scenes.

Contribution

The paper presents the first promptable multi-animal 3D reconstruction method based on SMAL+ and introduces Herd3D, a diverse dataset for training.

Findings

Achieves state-of-the-art results on multiple datasets.

Supports flexible prompts like keypoints and masks.

Effectively handles occlusions and crowded scenes.

Abstract

3D animal reconstruction in the wild remains challenging due to large species variation, frequent occlusions, and the prevalence of multi-animal scenes, while existing methods predominantly focus on single-animal settings. We present SAM 3D Animal, the first promptable framework for multi-animal 3D reconstruction from a single image. Built on the SMAL+ parametric animal model, our method jointly reconstructs multiple instances and supports flexible prompts in the form of keypoints and masks which enable more reliable disambiguation in crowded and occluded scenes. To train such a model, we further introduce Herd3D, a multi-animal 3D dataset containing over 5K images, designed to increase diversity in species, interactions, and occlusion patterns. Experiments on the Animal3D, APTv2, and Animal Kingdom datasets show that our framework achieves state-of-the-art results over both existing model-based and model-free methods, demonstrating a scalable and effective solution for prompt-driven animal 3D reconstruction in the wild.