A Survey of Robotic Navigation and Manipulation with Physics Simulators in the Era of Embodied AI

TL;DR

This survey reviews how physics simulators support robotic navigation and manipulation in Embodied AI, focusing on sim-to-real transfer challenges, features, and resources for researchers.

Contribution

It provides a comprehensive analysis of physics simulators' properties, features, and resources tailored for navigation and manipulation tasks in Embodied AI.

Findings

Analyzes properties of physics simulators relevant to Embodied AI

Provides benchmark datasets, metrics, and simulation platforms

Highlights cutting-edge methods like world models and geometric equivariance

Abstract

Navigation and manipulation are core capabilities in Embodied AI, yet training agents with these capabilities in the real world faces high costs and time complexity. Therefore, sim-to-real transfer has emerged as a key approach, yet the sim-to-real gap persists. This survey examines how physics simulators address this gap by analyzing their properties overlooked in previous surveys. We also analyze their features for navigation and manipulation tasks, along with hardware requirements. Additionally, we offer a resource with benchmark datasets, metrics, simulation platforms, and cutting-edge methods-such as world models and geometric equivariance-to help researchers select suitable tools while accounting for hardware constraints.