PhysPart: Physically Plausible Part Completion for Interactable Objects

TL;DR

This paper introduces PhysPart, a diffusion-based model for generating physically plausible 3D object parts that fit and move smoothly, advancing the modeling of interactable objects for applications like 3D printing and robotics.

Contribution

It proposes a novel diffusion model with physical constraints and geometric conditioning for realistic part completion, including dependent parts and sequential generation.

Findings

Outperforms existing methods on shape and physical plausibility metrics.

Introduces a new metric for physical plausibility based on motion success.

Demonstrates applications in 3D printing, robot manipulation, and sequential part generation.

Abstract

Interactable objects are ubiquitous in our daily lives. Recent advances in 3D generative models make it possible to automate the modeling of these objects, benefiting a range of applications from 3D printing to the creation of robot simulation environments. However, while significant progress has been made in modeling 3D shapes and appearances, modeling object physics, particularly for interactable objects, remains challenging due to the physical constraints imposed by inter-part motions. In this paper, we tackle the problem of physically plausible part completion for interactable objects, aiming to generate 3D parts that not only fit precisely into the object but also allow smooth part motions. To this end, we propose a diffusion-based part generation model that utilizes geometric conditioning through classifier-free guidance and formulates physical constraints as a set of stability and mobility losses to guide the sampling process. Additionally, we demonstrate the generation of dependent parts, paving the way toward sequential part generation for objects with complex part-whole hierarchies. Experimentally, we introduce a new metric for measuring physical plausibility based on motion success rates. Our model outperforms existing baselines over shape and physical metrics, especially those that do not adequately model physical constraints. We also demonstrate our applications in 3D printing, robot manipulation, and sequential part generation, showing our strength in realistic tasks with the demand for high physical plausibility.