CookBench: A Long-Horizon Embodied Planning Benchmark for Complex Cooking Scenarios

TL;DR

CookBench is a new benchmark designed for long-horizon, complex cooking tasks in embodied AI, featuring high-fidelity simulation, refined action granularity, and tools for high-level planning and decision-making.

Contribution

This paper introduces CookBench, a comprehensive benchmark with a realistic simulation environment and detailed action primitives for complex cooking scenarios, addressing limitations of existing benchmarks.

Findings

State-of-the-art models struggle with long-horizon tasks

CookBench enables evaluation of high-level planning in embodied AI

The benchmark is open-sourced for future research

Abstract

Embodied Planning is dedicated to the goal of creating agents capable of executing long-horizon tasks in complex physical worlds. However, existing embodied planning benchmarks frequently feature short-horizon tasks and coarse-grained action primitives. To address this challenge, we introduce CookBench, a benchmark for long-horizon planning in complex cooking scenarios. By leveraging a high-fidelity simulation environment built upon the powerful Unity game engine, we define frontier AI challenges in a complex, realistic environment. The core task in CookBench is designed as a two-stage process. First, in Intention Recognition, an agent needs to accurately parse a user's complex intent. Second, in Embodied Interaction, the agent should execute the identified cooking goal through a long-horizon, fine-grained sequence of physical actions. Unlike existing embodied planning benchmarks, we refine the action granularity to a spatial level that considers crucial operational information while abstracting away low-level robotic control. Besides, We provide a comprehensive toolset that encapsulates the simulator. Its unified API supports both macro-level operations, such as placing orders and purchasing ingredients, and a rich set of fine-grained embodied actions for physical interaction, enabling researchers to focus on high-level planning and decision-making. Furthermore, we present an in-depth analysis of state-of-the-art, closed-source Large Language Model and Vision-Language Model, revealing their major shortcomings and challenges posed by complex, long-horizon tasks. The full benchmark will be open-sourced to facilitate future research.