Towards Tangible Immersion for Cobot Programming-by-Demonstration: Visual, Tactile and Haptic Interfaces for Mixed-Reality Cobot Automation in Semiconductor Manufacturing

TL;DR

This paper introduces an embodiment-independent framework for reactive manipulation in robotic systems, enabling knowledge transfer across different platforms while maintaining reactive capabilities, demonstrated through object transport tasks.

Contribution

It proposes a novel abstraction layer of control primitives that allows modeling and executing reactive manipulation actions across various robotic embodiments.

Findings

Successfully transferred manipulation primitives between different robotic platforms.

Demonstrated robustness in handling unknown objects during box emptying tasks.

Validated the embodiment independence through experiments on multiple platforms.

Abstract

Sensor-based reactive and hybrid approaches have proven a promising line of study to address imperfect knowledge in grasping and manipulation. However the reactive approaches are usually tightly coupled to a particular embodiment making transfer of knowledge difficult. This paper proposes a paradigm for modeling and execution of reactive manipulation actions, which makes knowledge transfer to different embodiments possible while retaining the reactive capabilities of the embodiments. The proposed approach extends the idea of control primitives coordinated by a state machine by introducing an embodiment independent layer of abstraction. Abstract manipulation primitives constitute a vocabulary of atomic, embodiment independent actions, which can be coordinated using state machines to describe complex actions. To obtain embodiment specific models, the abstract state machines are automatically translated to embodiment specific models, such that full capabilities of each platform can be utilized. The strength of the manipulation primitives paradigm is demonstrated by developing a set of corresponding embodiment specific primitives for object transport, including a complex reactive grasping primitive. The robustness of the approach is experimentally studied in emptying of a box filled with several unknown objects. The embodiment independence is studied by performing a manipulation task on two different platforms using the same abstract description.