ViPS: Video-informed Pose Spaces for Auto-Rigged Meshes

TL;DR

ViPS introduces a novel, video-informed framework that learns a plausible, controllable pose space for auto-rigged meshes, enabling diverse, realistic animations without relying on extensive 4D datasets.

Contribution

It transfers generative video priors into a universal, shape-specific pose space for auto-rigged meshes, improving plausibility, diversity, and generalization in 3D pose modeling.

Findings

ViPS matches state-of-the-art models trained on 4D data in plausibility and diversity.

It enables sampling, inverse kinematics, and temporally coherent animations.

ViPS generalizes well to unseen species and skeletal topologies.

Abstract

Kinematic rigs provide a structured interface for articulating 3D meshes but lack any associated pose space, i.e., an explicit representation of the plausible manifold of joint configurations for a given mesh. Without such a pose space, stochastic sampling or manual manipulation of raw rig parameters easily results in semantic and/or geometric violations, such as anatomical hyperextension and non-physical self-intersections. We propose Video-informed Pose Spaces (ViPS), a feedforward framework that discovers the latent distribution of valid articulations for auto-rigged meshes by distilling motion priors from a pretrained video diffusion model. Unlike existing methods that rely on scarce, artist-authored 4D datasets, or focus on reconstructing instances of individual motions, ViPS transfers generative video model priors into a universal distribution over the given rig parameterization. Differentiable geometric validators applied to the skinned mesh enforce shape-specific integrity without requiring manual regularizers. Our feedforward model reveals a smooth, compact, and controllable pose space. This, in turn, supports sampling for diverse shape variations, manifold projection for inverse kinematics, and temporally coherent trajectories for animation and keyframing. Further, the distilled 3D pose samples serve as semantic proxies to guide video diffusion, effectively closing the loop between generative 2D priors and structured 3D kinematic control. Our evaluations show that ViPS, trained solely using video priors, matches the performance of state-of-the-art models trained on synthetic artist-created 4D data in both plausibility and diversity. Additionally, as a universal model, ViPS exhibits robust zero-shot generalization to out-of-distribution species and unseen skeletal topologies.