Task-Oriented Grasping Using Reinforcement Learning with a Contextual Reward Machine

TL;DR

This paper introduces a reinforcement learning framework with a Contextual Reward Machine for task-oriented grasping, significantly improving learning efficiency and success rates in both simulated and real-world robotic grasping tasks.

Contribution

It proposes a novel Contextual Reward Machine that decomposes tasks into sub-tasks with context-specific guidance, enhancing learning speed and success in robotic grasping.

Findings

Achieved 95% success rate in simulated grasping tasks.

Transferred the method to a real robot with 83.3% success.

Outperformed state-of-the-art methods in speed and accuracy.

Abstract

This paper presents a reinforcement learning framework that incorporates a Contextual Reward Machine for task-oriented grasping. The Contextual Reward Machine reduces task complexity by decomposing grasping tasks into manageable sub-tasks. Each sub-task is associated with a stage-specific context, including a reward function, an action space, and a state abstraction function. This contextual information enables efficient intra-stage guidance and improves learning efficiency by reducing the state-action space and guiding exploration within clearly defined boundaries. In addition, transition rewards are introduced to encourage or penalize transitions between stages which guides the model toward desirable stage sequences and further accelerates convergence. When integrated with the Proximal Policy Optimization algorithm, the proposed method achieved a 95% success rate across 1,000 simulated grasping tasks encompassing diverse objects, affordances, and grasp topologies. It outperformed the state-of-the-art methods in both learning speed and success rate. The approach was transferred to a real robot, where it achieved a success rate of 83.3% in 60 grasping tasks over six affordances. These experimental results demonstrate superior accuracy, data efficiency, and learning efficiency. They underscore the model's potential to advance task-oriented grasping in both simulated and real-world settings.