Tactile Tool Manipulation

TL;DR

This paper introduces a tactile sensor-based closed-loop control system for robotic tool manipulation, enabling robots to perform complex, reactive tasks similar to humans, by integrating pose estimation and model predictive control.

Contribution

It presents a novel closed-loop manipulation framework using tactile sensing and MPC, improving robot dexterity in unstructured environments.

Findings

Successful manipulation of various objects and tools

Robust performance under unexpected contacts

Enhanced dexterity with tactile feedback

Abstract

Humans can effortlessly perform very complex, dexterous manipulation tasks by reacting to sensor observations. In contrast, robots can not perform reactive manipulation and they mostly operate in open-loop while interacting with their environment. Consequently, the current manipulation algorithms either are inefficient in performance or can only work in highly structured environments. In this paper, we present closed-loop control of a complex manipulation task where a robot uses a tool to interact with objects. Manipulation using a tool leads to complex kinematics and contact constraints that need to be satisfied for generating feasible manipulation trajectories. We first present an open-loop controller design using Non-Linear Programming (NLP) that satisfies these constraints. In order to design a closed-loop controller, we present a pose estimator of objects and tools using tactile sensors. Using our tactile estimator, we design a closed-loop controller based on Model Predictive Control (MPC). The proposed algorithm is verified using a 6 DoF manipulator on tasks using a variety of objects and tools. We verify that our closed-loop controller can successfully perform tool manipulation under several unexpected contacts. Video summarizing this work and hardware experiments are found https://youtu.be/VsClK04qDhk.