Cook2LTL: Translating Cooking Recipes to LTL Formulae using Large Language Models

TL;DR

Cook2LTL leverages large language models and linear temporal logic to translate complex cooking recipes into robot-executable plans, reducing API calls, latency, and costs in simulated kitchen environments.

Contribution

This work introduces Cook2LTL, a novel system that combines LLMs with LTL formalism and a dynamic action library to translate recipes into robot plans efficiently.

Findings

Significant reduction in API calls (-51%)

Latency decreased by (-59%)

Cost lowered by (-42%)

Abstract

Cooking recipes are challenging to translate to robot plans as they feature rich linguistic complexity, temporally-extended interconnected tasks, and an almost infinite space of possible actions. Our key insight is that combining a source of cooking domain knowledge with a formalism that captures the temporal richness of cooking recipes could enable the extraction of unambiguous, robot-executable plans. In this work, we use Linear Temporal Logic (LTL) as a formal language expressive enough to model the temporal nature of cooking recipes. Leveraging a pretrained Large Language Model (LLM), we present Cook2LTL, a system that translates instruction steps from an arbitrary cooking recipe found on the internet to a set of LTL formulae, grounding high-level cooking actions to a set of primitive actions that are executable by a manipulator in a kitchen environment. Cook2LTL makes use of a caching scheme that dynamically builds a queryable action library at runtime. We instantiate Cook2LTL in a realistic simulation environment (AI2-THOR), and evaluate its performance across a series of cooking recipes. We demonstrate that our system significantly decreases LLM API calls (-51%), latency (-59%), and cost (-42%) compared to a baseline that queries the LLM for every newly encountered action at runtime.