Approximate Task Tree Retrieval in a Knowledge Network for Robotic Cooking

TL;DR

This paper presents a method for generating and modifying robotic task plans using a knowledge graph called FOON, enabling robots to adapt to new recipes with unseen ingredients and states, achieving 76% accuracy.

Contribution

It introduces a novel approach to adapt task plans in a knowledge graph for robotic cooking, allowing for the creation of new recipes with unseen ingredients and states.

Findings

Task trees can be generated for new recipes with unseen ingredients.

The system achieves 76% correctness in depicting new ingredient combinations.

Semantic similarity aids in enriching FOON with novel recipes.

Abstract

Flexible task planning continues to pose a difficult challenge for robots, where a robot is unable to creatively adapt their task plans to new or unseen problems, which is mainly due to the limited knowledge it has about its actions and world. Motivated by a human's ability to adapt, we explore how task plans from a knowledge graph, known as the Functional Object- Oriented Network (FOON), can be generated for novel problems requiring concepts that are not readily available to the robot in its knowledge base. Knowledge from 140 cooking recipes are structured in a FOON knowledge graph, which is used for acquiring task plan sequences known as task trees. Task trees can be modified to replicate recipes in a FOON knowledge graph format, which can be useful for enriching FOON with new recipes containing unknown object and state combinations, by relying upon semantic similarity. We demonstrate the power of task tree generation to create task trees with never-before-seen ingredient and state combinations as seen in recipes from the Recipe1M+ dataset, with which we evaluate the quality of the trees based on how accurately they depict newly added ingredients. Our experimental results show that our system is able to provide task sequences with 76% correctness.