Economical Precise Manipulation and Auto Eye-Hand Coordination with Binocular Visual Reinforcement Learning

TL;DR

This paper introduces Binocular Alignment Learning (BAL), a cost-effective method enabling precise robotic manipulation and eye-hand coordination using only low-cost webcams, trained in simulation and effective in real-world scenarios.

Contribution

The paper presents a novel approach for automatic eye-hand calibration and manipulation using weakly calibrated webcams, reducing costs and complexity in robotic systems.

Findings

Achieves competitive manipulation accuracy with low-cost webcams

Successfully transfers simulation training to real robots (sim2real)

Reduces need for expensive sensors and calibration procedures

Abstract

Precision robotic manipulation tasks (insertion, screwing, precisely pick, precisely place) are required in many scenarios. Previous methods achieved good performance on such manipulation tasks. However, such methods typically require tedious calibration or expensive sensors. 3D/RGB-D cameras and torque/force sensors add to the cost of the robotic application and may not always be economical. In this work, we aim to solve these but using only weak-calibrated and low-cost webcams. We propose Binocular Alignment Learning (BAL), which could automatically learn the eye-hand coordination and points alignment capabilities to solve the four tasks. Our work focuses on working with unknown eye-hand coordination and proposes different ways of performing eye-in-hand camera calibration automatically. The algorithm was trained in simulation and used a practical pipeline to achieve sim2real and test it on the real robot. Our method achieves a competitively good result with minimal cost on the four tasks.