A Reinforcement Learning Approach to Non-prehensile Manipulation through Sliding

TL;DR

This paper presents a reinforcement learning framework for non-prehensile sliding manipulation, enabling robots to adapt to different surfaces and transfer learned skills from simulation to real-world applications.

Contribution

It introduces a DDPG-based approach with online friction estimation, enhancing adaptability and robustness in non-prehensile sliding tasks.

Findings

Effective generalization across varying distances

Successful adaptation to different surface friction properties

Zero-shot sim-to-real transfer achieved

Abstract

Although robotic applications increasingly demand versatile and dynamic object handling, most existing techniques are predominantly focused on grasp-based manipulation, limiting their applicability in non-prehensile tasks. To address this need, this study introduces a Deep Deterministic Policy Gradient (DDPG) reinforcement learning framework for efficient non-prehensile manipulation, specifically for sliding an object on a surface. The algorithm generates a linear trajectory by precisely controlling the acceleration of a robotic arm rigidly coupled to the horizontal surface, enabling the relative manipulation of an object as it slides on top of the surface. Furthermore, two distinct algorithms have been developed to estimate the frictional forces dynamically during the sliding process. These algorithms provide online friction estimates after each action, which are fed back into the actor model as critical feedback after each action. This feedback mechanism enhances the policy's adaptability and robustness, ensuring more precise control of the platform's acceleration in response to varying surface condition. The proposed algorithm is validated through simulations and real-world experiments. Results demonstrate that the proposed framework effectively generalizes sliding manipulation across varying distances and, more importantly, adapts to different surfaces with diverse frictional properties. Notably, the trained model exhibits zero-shot sim-to-real transfer capabilities.