Formulating Intuitive Stack-of-Tasks using Visuo-Tactile Perception for Collaborative Human-Robot Fine Manipulation

TL;DR

This paper introduces an intuitive stack-of-tasks framework for collaborative human-robot manipulation, integrating visuo-tactile perception to enhance adaptability, safety, and coordination in close-proximity tasks.

Contribution

It presents a novel visuo-tactile augmented stack-of-tasks formulation that considers human posture and task progression for improved human-robot collaboration.

Findings

Effective perception of environment and human posture using depth cameras and tactile sensors.

Improved task coordination and adaptability in assembly/disassembly tasks.

Enhanced safety and efficiency in human-robot collaborative manipulation.

Abstract

Enabling robots to work in close proximity to humans necessitates a control framework that does not only incorporate multi-sensory information for autonomous and coordinated interactions but also has perceptive task planning to ensure an adaptable and flexible collaborative behaviour. In this research, an intuitive stack-of-tasks (iSoT) formulation is proposed, that defines the robot's actions by considering the human-arm postures and the task progression. The framework is augmented with visuo-tactile information to effectively perceive the collaborative environment and intuitively switch between the planned sub-tasks. The visual feedback from depth cameras monitors and estimates the objects' poses and human-arm postures, while the tactile data provides the exploration skills to detect and maintain the desired contacts to avoid object slippage. To evaluate the performance, effectiveness and usability of the proposed framework, assembly and disassembly tasks, performed by the human-human and human-robot partners, are considered and analyzed using distinct evaluation metrics i.e, approach adaptation, grasp correction, task coordination latency, cumulative posture deviation, and task repeatability.