Programming Manipulators by Instructions

TL;DR

This paper introduces an instructions-based framework for robot programming using simple commands, behavior trees, and a knowledge graph, enabling remote interaction and skill reuse in manipulation tasks.

Contribution

It presents a novel approach combining scripting commands, behavior trees, and knowledge graphs for flexible, remote robot programming and skill transfer.

Findings

Successful simulation of pick and place tasks

Enables remote programming without physical proximity

Supports reuse and adaptation of skills

Abstract

We propose an instructions-based approach for robot programming where the programmer interacts with the robot by issuing simple commands in a scripting language, like python. Internally, these commands make use of pre-programmed motion and manipulation skills coordinated by a behaviour tree task controller. A knowledge graph keeps track of the state of the robot and the environment and of all the instructions given to the robot by the programmer. This allows to easily transform sequences of instructions into new skills that can be reused in the same or in other tasks. An advantage of this approach is that the programmer does not need to be located physically next to the robot, but can work remotely, either with a physical robot or with a digital twin. To demonstrate the concept, we show an interactive simulation of a robot manipulator in a pick and place scenario.