An Efficient Image-to-Image Translation HourGlass-based Architecture for Object Pushing Policy Learning

TL;DR

This paper introduces an Hourglass-based image-to-image translation architecture for learning object pushing policies with Deep Q-Networks, improving learning speed and performance in robotic pushing tasks involving unknown object dynamics.

Contribution

It proposes framing pushing policy learning as an image-to-image translation problem using an Hourglass architecture, integrating environment change prediction and position encoding.

Findings

Faster learning convergence in simulation

Higher success rates in pushing tasks

Effective handling of unknown object dynamics

Abstract

Humans effortlessly solve pushing tasks in everyday life but unlocking these capabilities remains a challenge in robotics because physics models of these tasks are often inaccurate or unattainable. State-of-the-art data-driven approaches learn to compensate for these inaccuracies or replace the approximated physics models altogether. Nevertheless, approaches like Deep Q-Networks (DQNs) suffer from local optima in large state-action spaces. Furthermore, they rely on well-chosen deep learning architectures and learning paradigms. In this paper, we propose to frame the learning of pushing policies (where to push and how) by DQNs as an image-to-image translation problem and exploit an Hourglass-based architecture. We present an architecture combining a predictor of which pushes lead to changes in the environment with a state-action value predictor dedicated to the pushing task. Moreover, we investigate positional information encoding to learn position-dependent policy behaviors. We demonstrate in simulation experiments with a UR5 robot arm that our overall architecture helps the DQN learn faster and achieve higher performance in a pushing task involving objects with unknown dynamics.