Zero-shot sim-to-real transfer of tactile control policies for aggressive swing-up manipulation

TL;DR

This paper demonstrates that a tactile sensor-based robotic system can perform pole swing-up tasks across various physical attributes without prior object knowledge, using a novel simulation and learned feedback policy.

Contribution

It introduces a new simulation for tactile interaction and a zero-shot transfer policy for dynamic pole swing-up without prior physical object knowledge.

Findings

Successful swing-up of diverse poles in real-world tests

First use of tactile feedback policies for closed-loop pole manipulation

Policy generalizes across different pole properties without adaptation

Abstract

This paper aims to show that robots equipped with a vision-based tactile sensor can perform dynamic manipulation tasks without prior knowledge of all the physical attributes of the objects to be manipulated. For this purpose, a robotic system is presented that is able to swing up poles of different masses, radii and lengths, to an angle of 180 degrees, while relying solely on the feedback provided by the tactile sensor. This is achieved by developing a novel simulator that accurately models the interaction of a pole with the soft sensor. A feedback policy that is conditioned on a sensory observation history, and which has no prior knowledge of the physical features of the pole, is then learned in the aforementioned simulation. When evaluated on the physical system, the policy is able to swing up a wide range of poles that differ significantly in their physical attributes without further adaptation. To the authors' knowledge, this is the first work where a feedback policy from high-dimensional tactile observations is used to control the swing-up manipulation of poles in closed-loop.