Online Object and Task Learning via Human Robot Interaction

TL;DR

This paper presents a robotic system capable of incremental learning of objects and tasks through human interaction, integrating deep learning, user interfaces, and hybrid control for adaptable robot behavior.

Contribution

It introduces a novel deep learning-based object recognition module and an intuitive interface for teaching new objects and tasks in real-time.

Findings

System was a finalist in the KUKA Innovation Award

Successfully demonstrated incremental object learning and task specification

Integrated hybrid force-vision control for unstructured surface interaction

Abstract

This work describes the development of a robotic system that acquires knowledge incrementally through human interaction where new tools and motions are taught on the fly. The robotic system developed was one of the five finalists in the KUKA Innovation Award competition and demonstrated during the Hanover Messe 2018 in Germany. The main contributions of the system are a) a novel incremental object learning module - a deep learning based localization and recognition system - that allows a human to teach new objects to the robot, b) an intuitive user interface for specifying 3D motion task associated with the new object, c) a hybrid force-vision control module for performing compliant motion on an unstructured surface. This paper describes the implementation and integration of the main modules of the system and summarizes the lessons learned from the competition.