ManiSkill: Generalizable Manipulation Skill Benchmark with Large-Scale Demonstrations
Tongzhou Mu, Zhan Ling, Fanbo Xiang, Derek Yang, Xuanlin Li, Stone, Tao, Zhiao Huang, Zhiwei Jia, Hao Su

TL;DR
ManiSkill is a comprehensive benchmark for object manipulation that features diverse 3D objects, realistic physics, and extensive demonstrations, aiming to advance generalizable manipulation models and facilitate research in 3D vision and learning-from-demonstrations.
Contribution
The paper introduces ManiSkill, a large-scale, diverse manipulation benchmark with high-quality demonstrations and a full-physics simulator, supporting both policy learning and learning-from-demonstrations.
Findings
Benchmark includes over 36,000 successful trajectories.
Provides diverse 3D assets with topological and geometric variations.
Open-sourced code and baseline algorithms are available.
Abstract
Object manipulation from 3D visual inputs poses many challenges on building generalizable perception and policy models. However, 3D assets in existing benchmarks mostly lack the diversity of 3D shapes that align with real-world intra-class complexity in topology and geometry. Here we propose SAPIEN Manipulation Skill Benchmark (ManiSkill) to benchmark manipulation skills over diverse objects in a full-physics simulator. 3D assets in ManiSkill include large intra-class topological and geometric variations. Tasks are carefully chosen to cover distinct types of manipulation challenges. Latest progress in 3D vision also makes us believe that we should customize the benchmark so that the challenge is inviting to researchers working on 3D deep learning. To this end, we simulate a moving panoramic camera that returns ego-centric point clouds or RGB-D images. In addition, we would like…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Code & Models
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
Topics3D Shape Modeling and Analysis · Advanced Vision and Imaging · Robot Manipulation and Learning
