ArtVIP: Articulated Digital Assets of Visual Realism, Modular Interaction, and Physical Fidelity for Robot Learning

TL;DR

ArtVIP introduces a high-quality, open-source dataset of digitally realistic, physically accurate articulated objects with modular interactions, enhancing robot learning simulation and bridging the gap to real-world applications.

Contribution

It provides a comprehensive, realistic, and physically faithful dataset with embedded interaction behaviors and annotations, addressing limitations of existing datasets for robot training.

Findings

Demonstrated improved sim-to-real transfer in robot learning tasks.

Validated dataset's visual and physical fidelity through feature-map visualization and motion capture.

Enabled effective imitation and reinforcement learning experiments.

Abstract

Robot learning increasingly relies on simulation to advance complex ability such as dexterous manipulations and precise interactions, necessitating high-quality digital assets to bridge the sim-to-real gap. However, existing open-source articulated-object datasets for simulation are limited by insufficient visual realism and low physical fidelity, which hinder their utility for training models mastering robotic tasks in real world. To address these challenges, we introduce ArtVIP, a comprehensive open-source dataset comprising high-quality digital-twin articulated objects, accompanied by indoor-scene assets. Crafted by professional 3D modelers adhering to unified standards, ArtVIP ensures visual realism through precise geometric meshes and high-resolution textures, while physical fidelity is achieved via fine-tuned dynamic parameters. Meanwhile, the dataset pioneers embedded modular interaction behaviors within assets and pixel-level affordance annotations. Feature-map visualization and optical motion capture are employed to quantitatively demonstrate ArtVIP's visual and physical fidelity, with its applicability validated across imitation learning and reinforcement learning experiments. Provided in USD format with detailed production guidelines, ArtVIP is fully open-source, benefiting the research community and advancing robot learning research. Our project is at https://x-humanoid-artvip.github.io/ .