Long-Horizon Planning and Execution with Functional Object-Oriented Networks

TL;DR

This paper presents a method to convert functional object-oriented networks (FOON) into executable plans for robots by transforming FOON into PDDL and integrating with existing planners, demonstrated on long-horizon tasks in simulation.

Contribution

It introduces a novel pipeline that automatically transforms FOON into PDDL for hierarchical planning and execution in robotic tasks, bridging the gap between abstract knowledge and practical robot actions.

Findings

Successful execution of long-horizon tasks in simulation

Extension of learned action contexts to new scenarios

Effective transformation of FOON into PDDL for planning

Abstract

Following work on joint object-action representations, functional object-oriented networks (FOON) were introduced as a knowledge graph representation for robots. A FOON contains symbolic concepts useful to a robot's understanding of tasks and its environment for object-level planning. Prior to this work, little has been done to show how plans acquired from FOON can be executed by a robot, as the concepts in a FOON are too abstract for execution. We thereby introduce the idea of exploiting object-level knowledge as a FOON for task planning and execution. Our approach automatically transforms FOON into PDDL and leverages off-the-shelf planners, action contexts, and robot skills in a hierarchical planning pipeline to generate executable task plans. We demonstrate our entire approach on long-horizon tasks in CoppeliaSim and show how learned action contexts can be extended to never-before-seen scenarios.