DISCOVERSE: Efficient Robot Simulation in Complex High-Fidelity Environments

TL;DR

Discoverse is a modular, open-source 3D simulation framework that enables high-fidelity, realistic robot learning environments, significantly improving zero-shot Sim2Real transfer performance for complex robotic tasks.

Contribution

It introduces a unified simulation platform combining Gaussian Splatting and MuJoCo for realistic, scalable robot simulation with extensive support for sensor modalities and assets.

Findings

Achieves state-of-the-art zero-shot Sim2Real transfer in experiments

Supports massively parallel simulation of multiple sensors and physics

Enables large-scale robot learning and complex benchmarks

Abstract

We present the first unified, modular, open-source 3DGS-based simulation framework for Real2Sim2Real robot learning. It features a holistic Real2Sim pipeline that synthesizes hyper-realistic geometry and appearance of complex real-world scenarios, paving the way for analyzing and bridging the Sim2Real gap. Powered by Gaussian Splatting and MuJoCo, Discoverse enables massively parallel simulation of multiple sensor modalities and accurate physics, with inclusive supports for existing 3D assets, robot models, and ROS plugins, empowering large-scale robot learning and complex robotic benchmarks. Through extensive experiments on imitation learning, Discoverse demonstrates state-of-the-art zero-shot Sim2Real transfer performance compared to existing simulators. For code and demos: https://air-discoverse.github.io/.