DiffPose-Animal: A Language-Conditioned Diffusion Framework for Animal Pose Estimation

TL;DR

DiffPose-Animal introduces a diffusion-based, language-guided framework for animal pose estimation, effectively handling diverse species, occlusions, and limited data by integrating semantic priors and progressive refinement.

Contribution

It pioneers the use of diffusion models combined with language-derived semantic priors for top-down animal pose estimation, enhancing robustness and generalization.

Findings

Outperforms existing methods on public datasets.

Effectively handles occlusion and sparse annotations.

Demonstrates strong generalization across diverse species.

Abstract

Animal pose estimation is a fundamental task in computer vision, with growing importance in ecological monitoring, behavioral analysis, and intelligent livestock management. Compared to human pose estimation, animal pose estimation is more challenging due to high interspecies morphological diversity, complex body structures, and limited annotated data. In this work, we introduce DiffPose-Animal, a novel diffusion-based framework for top-down animal pose estimation. Unlike traditional heatmap regression methods, DiffPose-Animal reformulates pose estimation as a denoising process under the generative framework of diffusion models. To enhance semantic guidance during keypoint generation, we leverage large language models (LLMs) to extract both global anatomical priors and local keypoint-wise semantics based on species-specific prompts. These textual priors are encoded and fused with image features via cross-attention modules to provide biologically meaningful constraints throughout the denoising process. Additionally, a diffusion-based keypoint decoder is designed to progressively refine pose predictions, improving robustness to occlusion and annotation sparsity. Extensive experiments on public animal pose datasets demonstrate the effectiveness and generalization capability of our method, especially under challenging scenarios with diverse species, cluttered backgrounds, and incomplete keypoints.