Split Deep Q-Learning for Robust Object Singulation

TL;DR

This paper introduces a novel Split Deep Q-Learning approach for robotic object singulation, enabling effective transfer from simulation to real-world environments and supporting modular policy enhancements.

Contribution

The paper proposes a Split DQN method that improves learning efficiency and modularity for robotic object singulation tasks using reinforcement learning.

Findings

Effective transfer of learned policies from simulation to real robots

Modular algorithm allows addition of new primitives without retraining

Split DQN enhances learning rate and algorithm modularity

Abstract

Extracting a known target object from a pile of other objects in a cluttered environment is a challenging robotic manipulation task encountered in many robotic applications. In such conditions, the target object touches or is covered by adjacent obstacle objects, thus rendering traditional grasping techniques ineffective. In this paper, we propose a pushing policy aiming at singulating the target object from its surrounding clutter, by means of lateral pushing movements of both the neighboring objects and the target object until sufficient 'grasping room' has been achieved. To achieve the above goal we employ reinforcement learning and particularly Deep Q-learning (DQN) to learn optimal push policies by trial and error. A novel Split DQN is proposed to improve the learning rate and increase the modularity of the algorithm. Experiments show that although learning is performed in a simulated environment the transfer of learned policies to a real environment is effective thanks to robust feature selection. Finally, we demonstrate that the modularity of the algorithm allows the addition of extra primitives without retraining the model from scratch.