MinePlanner: A Benchmark for Long-Horizon Planning in Large Minecraft Worlds

TL;DR

This paper introduces MinePlanner, a comprehensive benchmark for long-horizon planning in large Minecraft worlds, highlighting current planner limitations and guiding future improvements.

Contribution

It presents a new benchmark with 45 tasks for Minecraft planning, supporting automatic creation of propositional and numeric instances, and evaluates existing planners on these tasks.

Findings

State-of-the-art planners struggle with large-scale Minecraft tasks.

Current planners cannot handle instances with thousands of objects.

The benchmark reveals specific challenges and areas for future research.

Abstract

We propose a new benchmark for planning tasks based on the Minecraft game. Our benchmark contains 45 tasks overall, but also provides support for creating both propositional and numeric instances of new Minecraft tasks automatically. We benchmark numeric and propositional planning systems on these tasks, with results demonstrating that state-of-the-art planners are currently incapable of dealing with many of the challenges advanced by our new benchmark, such as scaling to instances with thousands of objects. Based on these results, we identify areas of improvement for future planners. Our framework is made available at https://github.com/IretonLiu/mine-pddl/.