MarketGen: A Scalable Simulation Platform with Auto-Generated Embodied Supermarket Environments

TL;DR

MarketGen is a scalable simulation platform that automatically generates realistic supermarket environments, enabling research on embodied AI agents for complex commercial tasks with diverse assets and benchmarks.

Contribution

It introduces a novel agent-based procedural scene generation framework supporting multi-modal inputs and a new benchmark for supermarket tasks.

Findings

Successful sim-to-real transfer of agents

Extensive dataset with 1100+ supermarket goods

Validation of platform through diverse experiments

Abstract

The development of embodied agents for complex commercial environments is hindered by a critical gap in existing robotics datasets and benchmarks, which primarily focus on household or tabletop settings with short-horizon tasks. To address this limitation, we introduce MarketGen, a scalable simulation platform with automatic scene generation for complex supermarket environments. MarketGen features a novel agent-based Procedural Content Generation (PCG) framework. It uniquely supports multi-modal inputs (text and reference images) and integrates real-world design principles to automatically generate complete, structured, and realistic supermarkets. We also provide an extensive and diverse 3D asset library with a total of 1100+ supermarket goods and parameterized facilities assets. Building on this generative foundation, we propose a novel benchmark for assessing supermarket agents, featuring two daily tasks in a supermarket: (1) Checkout Unloading: long-horizon tabletop tasks for cashier agents, and (2) In-Aisle Item Collection: complex mobile manipulation tasks for salesperson agents. We validate our platform and benchmark through extensive experiments, including the deployment of a modular agent system and successful sim-to-real transfer. MarketGen provides a comprehensive framework to accelerate research in embodied AI for complex commercial applications.