Automatically Improving Simulation Physics for Articulated Objects

TL;DR

This paper presents a multi-modal, simulator-in-the-loop method to automatically generate and refine physically realistic, interaction-ready articulated objects from incomplete 3D assets, improving simulation stability and policy performance.

Contribution

It introduces the concept of interaction-readiness, a quantitative evaluation framework, and a multi-modal refinement approach that leverages simulator feedback to enhance physical realism of articulated objects.

Findings

Refined objects exhibit more stable and realistic dynamics.

Object quality impacts simulation stability and policy effectiveness.

The method enables scalable construction of simulation-ready objects.

Abstract

Simulation is a central tool for scalable robot learning, but its effectiveness depends on the quality of object assets. While modern 3D datasets provide rich geometric and kinematic representations, they typically lack the physical properties required for stable and realistic interaction, requiring significant manual effort to construct simulation-ready articulated objects. In this thesis, we introduce interaction-readiness, which characterizes whether an object can be reliably simulated under manipulation. We propose a quantitative evaluation framework that decomposes interaction-readiness into measurable components, enabling systematic analysis of object quality and revealing failure modes not captured by conventional evaluation. We further present a multi-modal, simulator-in-the-loop approach for generating interaction-ready articulated objects from incomplete 3D assets. The method integrates geometric, visual, and semantic information to infer physical properties and refines them through iterative simulator feedback to improve physical consistency. Experiments across diverse articulated objects and manipulation tasks show that object quality directly impacts simulation stability, interaction behavior, and policy performance. Objects refined by our method exhibit more stable and realistic dynamics, enabling more reliable downstream learning and evaluation. Overall, this thesis demonstrates the importance of physical realism for articulated objects in simulation and introduces a practical multi-modal refinement approach, guided by simulator feedback, for constructing such objects at scale.