# Functional Object-Oriented Network for Manipulation Learning

## TL;DR

This paper introduces the functional object-oriented network (FOON), a structured knowledge representation that enables robots to understand and perform manipulation tasks by learning from observations and instructional videos.

## Contribution

The paper proposes FOON as a new graphical model for manipulation learning, including methods to learn, represent, and retrieve manipulation sequences from online videos.

## Key findings

- FOON effectively models object-function relationships in manipulation tasks.
- Robots can generate task-specific motion sequences using FOON.
- Demonstrations show FOON's adaptability in simulated environments.

## Abstract

This paper presents a novel structured knowledge representation called the functional object-oriented network (FOON) to model the connectivity of the functional-related objects and their motions in manipulation tasks. The graphical model FOON is learned by observing object state change and human manipulations with the objects. Using a well-trained FOON, robots can decipher a task goal, seek the correct objects at the desired states on which to operate, and generate a sequence of proper manipulation motions. The paper describes FOON's structure and an approach to form a universal FOON with extracted knowledge from online instructional videos. A graph retrieval approach is presented to generate manipulation motion sequences from the FOON to achieve a desired goal, demonstrating the flexibility of FOON in creating a novel and adaptive means of solving a problem using knowledge gathered from multiple sources. The results are demonstrated in a simulated environment to illustrate the motion sequences generated from the FOON to carry out the desired tasks.