CleanUpBench: Embodied Sweeping and Grasping Benchmark

TL;DR

CleanUpBench is a new benchmark for evaluating embodied mobile cleaning robots with sweeping and grasping capabilities in realistic indoor environments, addressing a gap between research and real-world deployment.

Contribution

It introduces a reproducible, extensible benchmark built on NVIDIA Isaac Sim for assessing embodied agents in structured cleaning tasks, including environments, evaluation metrics, and baseline agents.

Findings

Benchmark enables systematic evaluation of cleaning robots.

Baseline agents demonstrate the platform's utility for comparison.

Supports generalization assessment with procedural environments.

Abstract

Embodied AI benchmarks have advanced navigation, manipulation, and reasoning, but most target complex humanoid agents or large-scale simulations that are far from real-world deployment. In contrast, mobile cleaning robots with dual mode capabilities, such as sweeping and grasping, are rapidly emerging as realistic and commercially viable platforms. However, no benchmark currently exists that systematically evaluates these agents in structured, multi-target cleaning tasks, revealing a critical gap between academic research and real-world applications. We introduce CleanUpBench, a reproducible and extensible benchmark for evaluating embodied agents in realistic indoor cleaning scenarios. Built on NVIDIA Isaac Sim, CleanUpBench simulates a mobile service robot equipped with a sweeping mechanism and a six-degree-of-freedom robotic arm, enabling interaction with heterogeneous objects. The benchmark includes manually designed environments and one procedurally generated layout to assess generalization, along with a comprehensive evaluation suite covering task completion, spatial efficiency, motion quality, and control performance. To support comparative studies, we provide baseline agents based on heuristic strategies and map-based planning. CleanUpBench bridges the gap between low-level skill evaluation and full-scene testing, offering a scalable testbed for grounded, embodied intelligence in everyday settings.