A Road-map to Robot Task Execution with the Functional Object-Oriented Network

TL;DR

This paper presents a future development plan for the Functional Object-Oriented Network (FOON), a knowledge graph for robots, focusing on task planning and learning from demonstration to enhance robotic manipulation capabilities.

Contribution

It proposes a roadmap for developing FOON for real-world robot applications, including methods for knowledge acquisition from human demonstrations.

Findings

Preliminary ideas for creating FOON with robot and human collaboration.

Framework for augmenting FOON with new knowledge from demonstrations.

Potential for FOON to improve robot task understanding and execution.

Abstract

Following work on joint object-action representations, the functional object-oriented network (FOON) was introduced as a knowledge graph representation for robots. Taking the form of a bipartite graph, a FOON contains symbolic or high-level information that would be pertinent to a robot's understanding of its environment and tasks in a way that mirrors human understanding of actions. In this work, we outline a road-map for future development of FOON and its application in robotic systems for task planning as well as knowledge acquisition from demonstration. We propose preliminary ideas to show how a FOON can be created in a real-world scenario with a robot and human teacher in a way that can jointly augment existing knowledge in a FOON and teach a robot the skills it needs to replicate the demonstrated actions and solve a given manipulation problem.