Versatile and Generalizable Manipulation via Goal-Conditioned Reinforcement Learning with Grounded Object Detection

TL;DR

This paper introduces a goal-conditioned reinforcement learning framework that leverages pre-trained object detection models to enable versatile and generalizable robotic reach and grasp capabilities, demonstrating high success rates and faster learning.

Contribution

It integrates pre-trained object detection into goal-conditioned RL, enabling object-agnostic manipulation with improved generalization and efficiency.

Findings

Achieved ~90% success rate in grasping diverse objects

Faster convergence to higher returns in simulated tasks

Effective in both in-distribution and out-of-distribution scenarios

Abstract

General-purpose robotic manipulation, including reach and grasp, is essential for deployment into households and workspaces involving diverse and evolving tasks. Recent advances propose using large pre-trained models, such as Large Language Models and object detectors, to boost robotic perception in reinforcement learning. These models, trained on large datasets via self-supervised learning, can process text prompts and identify diverse objects in scenes, an invaluable skill in RL where learning object interaction is resource-intensive. This study demonstrates how to integrate such models into Goal-Conditioned Reinforcement Learning to enable general and versatile robotic reach and grasp capabilities. We use a pre-trained object detection model to enable the agent to identify the object from a text prompt and generate a mask for goal conditioning. Mask-based goal conditioning provides object-agnostic cues, improving feature sharing and generalization. The effectiveness of the proposed framework is demonstrated in a simulated reach-and-grasp task, where the mask-based goal conditioning consistently maintains a $\sim$90\% success rate in grasping both in and out-of-distribution objects, while also ensuring faster convergence to higher returns.