Towards Affordance-Aware Articulation Synthesis for Rigged Objects

TL;DR

This paper introduces A3Syn, a novel method for synthesizing realistic, affordance-aware postures for rigged objects using a diffusion model and semantic alignment, reducing manual effort and handling open-domain rigs.

Contribution

A3Syn is the first approach to synthesize affordance-aware articulations for open-domain rigged objects without topological assumptions, utilizing diffusion models and semantic correspondence.

Findings

Synthesizes plausible affordance postures in minutes.

Handles diverse open-domain rigged objects and scenes.

Stable convergence demonstrated across various scenarios.

Abstract

Rigged objects are commonly used in artist pipelines, as they can flexibly adapt to different scenes and postures. However, articulating the rigs into realistic affordance-aware postures (e.g., following the context, respecting the physics and the personalities of the object) remains time-consuming and heavily relies on human labor from experienced artists. In this paper, we tackle the novel problem and design A3Syn. With a given context, such as the environment mesh and a text prompt of the desired posture, A3Syn synthesizes articulation parameters for arbitrary and open-domain rigged objects obtained from the Internet. The task is incredibly challenging due to the lack of training data, and we do not make any topological assumptions about the open-domain rigs. We propose using 2D inpainting diffusion model and several control techniques to synthesize in-context affordance information. Then, we develop an efficient bone correspondence alignment using a combination of differentiable rendering and semantic correspondence. A3Syn has stable convergence, completes in minutes, and synthesizes plausible affordance on different combinations of in-the-wild object rigs and scenes.