The Proof is in the Almond Cookies

TL;DR

This paper introduces a narrative-based computational approach to understanding cooking recipes, enabling robots to interpret and execute recipes by mimicking human sense-making, with applications in assistive and professional kitchens.

Contribution

It proposes a novel narrative modeling method for recipe understanding that integrates language, ontologies, and mental simulation, advancing AI support for cooking tasks.

Findings

Effective handling of recipe language challenges like zero anaphora

Improved robot planning efficiency in cooking tasks

Recipe annotations become language-independent

Abstract

This paper presents a case study on how to process cooking recipes (and more generally, how-to instructions) in a way that makes it possible for a robot or artificial cooking assistant to support human chefs in the kitchen. Such AI assistants would be of great benefit to society, as they can help to sustain the autonomy of aging adults or people with a physical impairment, or they may reduce the stress in a professional kitchen. We propose a novel approach to computational recipe understanding that mimics the human sense-making process, which is narrative-based. Using an English recipe for almond crescent cookies as illustration, we show how recipes can be modelled as rich narrative structures by integrating various knowledge sources such as language processing, ontologies, and mental simulation. We show how such narrative structures can be used for (a) dealing with the challenges of recipe language, such as zero anaphora, (b) optimizing a robot's planning process, (c) measuring how well an AI system understands its current tasks, and (d) allowing recipe annotations to become language-independent.