Sim-to-Real Model-Based and Model-Free Deep Reinforcement Learning for Tactile Pushing

TL;DR

This paper introduces a tactile sensing-based deep reinforcement learning approach for object pushing that generalizes well to unseen objects and scenarios, demonstrating effective sim-to-real transfer without domain randomization.

Contribution

It presents a novel goal-conditioned tactile pushing method using both model-free and model-based RL, achieving reliable real-world manipulation with limited training data.

Findings

Model-free policy can outperform model-based planner with more training samples.

Policies trained on a single object generalize to unseen objects and scenarios.

Effective sim-to-real transfer achieved without domain randomization.

Abstract

Object pushing presents a key non-prehensile manipulation problem that is illustrative of more complex robotic manipulation tasks. While deep reinforcement learning (RL) methods have demonstrated impressive learning capabilities using visual input, a lack of tactile sensing limits their capability for fine and reliable control during manipulation. Here we propose a deep RL approach to object pushing using tactile sensing without visual input, namely tactile pushing. We present a goal-conditioned formulation that allows both model-free and model-based RL to obtain accurate policies for pushing an object to a goal. To achieve real-world performance, we adopt a sim-to-real approach. Our results demonstrate that it is possible to train on a single object and a limited sample of goals to produce precise and reliable policies that can generalize to a variety of unseen objects and pushing scenarios without domain randomization. We experiment with the trained agents in harsh pushing conditions, and show that with significantly more training samples, a model-free policy can outperform a model-based planner, generating shorter and more reliable pushing trajectories despite large disturbances. The simplicity of our training environment and effective real-world performance highlights the value of rich tactile information for fine manipulation. Code and videos are available at https://sites.google.com/view/tactile-rl-pushing/.