Visuo-Tactile Manipulation Planning Using Reinforcement Learning with Affordance Representation

TL;DR

This paper introduces a reinforcement learning framework that integrates multisensory feedback and a deep affordance model to improve robotic object manipulation in unstructured environments.

Contribution

It presents a novel multisensory affordance-based reinforcement learning approach for manipulation planning, combining vision and touch for better perceptual understanding.

Findings

Outperforms existing methods in accuracy and efficiency

Effectively predicts manipulable regions using multisensory data

Demonstrates robustness in unstructured environments

Abstract

Robots are increasingly expected to manipulate objects in ever more unstructured environments where the object properties have high perceptual uncertainty from any single sensory modality. This directly impacts successful object manipulation. In this work, we propose a reinforcement learning-based motion planning framework for object manipulation which makes use of both on-the-fly multisensory feedback and a learned attention-guided deep affordance model as perceptual states. The affordance model is learned from multiple sensory modalities, including vision and touch (tactile and force/torque), which is designed to predict and indicate the manipulable regions of multiple affordances (i.e., graspability and pushability) for objects with similar appearances but different intrinsic properties (e.g., mass distribution). A DQN-based deep reinforcement learning algorithm is then trained to select the optimal action for successful object manipulation. To validate the performance of the proposed framework, our method is evaluated and benchmarked using both an open dataset and our collected dataset. The results show that the proposed method and overall framework outperform existing methods and achieve better accuracy and higher efficiency.