UnrealZoo: Enriching Photo-realistic Virtual Worlds for Embodied AI
Fangwei Zhong, Kui Wu, Churan Wang, Hao Chen, Hai Ci, Zhoujun Li, Yizhou Wang

TL;DR
UnrealZoo offers a diverse set of photo-realistic virtual environments and tools to advance embodied AI research, highlighting the importance of environmental diversity and identifying current challenges in open-world scenarios.
Contribution
UnrealZoo provides a large collection of realistic virtual worlds and enhanced APIs, facilitating efficient data collection, environment augmentation, and benchmarking for embodied AI.
Findings
Environmental diversity improves RL generalization.
Embodied agents struggle with unstructured terrain and unseen morphologies.
Latency issues affect dynamic object interaction.
Abstract
We introduce UnrealZoo, a collection of over 100 photo-realistic 3D virtual worlds built on Unreal Engine, designed to reflect the complexity and variability of open-world environments. We also provide a rich variety of playable entities, including humans, animals, robots, and vehicles for embodied AI research. We extend UnrealCV with optimized APIs and tools for data collection, environment augmentation, distributed training, and benchmarking. These improvements achieve significant improvements in the efficiency of rendering and communication, enabling advanced applications such as multi-agent interactions. Our experimental evaluation across visual navigation and tracking tasks reveals two key insights: 1) environmental diversity provides substantial benefits for developing generalizable reinforcement learning (RL) agents, and 2) current embodied agents face persistent challenges in…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsHuman Pose and Action Recognition · Innovative Human-Technology Interaction · Virtual Reality Applications and Impacts
