GRUtopia: Dream General Robots in a City at Scale
Hanqing Wang, Jiahe Chen, Wensi Huang, Qingwei Ben, Tai Wang, Boyu Mi,, Tao Huang, Siheng Zhao, Yilun Chen, Sizhe Yang, Peizhou Cao, Wenye Yu, Zichao, Ye, Jialun Li, Junfeng Long, Zirui Wang, Huiling Wang, Ying Zhao, Zhongying, Tu, Yu Qiao, Dahua Lin, Jiangmiao Pang

TL;DR
GRUtopia introduces a large-scale, city-like simulation environment with diverse scenes and social interaction capabilities, aiming to advance Embodied AI research through scalable data and comprehensive benchmarking.
Contribution
It presents the first city-scale interactive simulation platform with extensive annotated scenes, social NPCs driven by LLMs, and a benchmark for evaluating various robot tasks.
Findings
100k annotated scenes covering 89 categories
A social NPC system for interaction and task generation
Benchmark focusing on legged robots and complex tasks
Abstract
Recent works have been exploring the scaling laws in the field of Embodied AI. Given the prohibitive costs of collecting real-world data, we believe the Simulation-to-Real (Sim2Real) paradigm is a crucial step for scaling the learning of embodied models. This paper introduces project GRUtopia, the first simulated interactive 3D society designed for various robots. It features several advancements: (a) The scene dataset, GRScenes, includes 100k interactive, finely annotated scenes, which can be freely combined into city-scale environments. In contrast to previous works mainly focusing on home, GRScenes covers 89 diverse scene categories, bridging the gap of service-oriented environments where general robots would be initially deployed. (b) GRResidents, a Large Language Model (LLM) driven Non-Player Character (NPC) system that is responsible for social interaction, task generation, and…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Code & Models
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsModular Robots and Swarm Intelligence · Robotic Path Planning Algorithms · Opportunistic and Delay-Tolerant Networks
