Virtual-force Based Visual Servo for Multiple Peg-in-Hole Assembly with Tightly Coupled Multi-Manipulator

TL;DR

This paper introduces a collaborative visual servo control framework for multi-manipulator peg-in-hole assembly, utilizing monocular cameras and neural networks to improve accuracy and success rates in complex robotic assembly tasks.

Contribution

It proposes a novel virtual-force based visual servo method integrating multi-manipulator visual features, trained neural networks for state classification and positioning, and demonstrates high success in precise dual-manipulator assembly.

Findings

Achieved 100% success rate in dual-manipulator peg-in-hole tasks with 0.2 mm clearance.

Improved classification accuracy and positioning precision by considering hole appearance.

Robust to camera calibration errors in complex assembly scenarios.

Abstract

Multiple Peg-in-Hole (MPiH) assembly is one of the fundamental tasks in robotic assembly. In the MPiH tasks for large-size parts, it is challenging for a single manipulator to simultaneously align multiple distant pegs and holes, necessitating tightly coupled multi-manipulator systems. For such MPiH tasks using tightly coupled multiple manipulators, we propose a collaborative visual servo control framework that uses only the monocular in-hand cameras of each manipulator to reduce positioning errors. Initially, we train a state classification neural network and a positioning neural network. The former divides the states of the peg and hole in the image into three categories: obscured, separated, and overlapped, while the latter determines the position of the peg and hole in the image. Based on these findings, we propose a method to integrate the visual features of multiple manipulators using virtual forces, which can naturally combine with the cooperative controller of the multi-manipulator system. To generalize our approach to holes of different appearances, we varied the appearance of the holes during the dataset generation process. The results confirm that by considering the appearance of the holes, classification accuracy and positioning precision can be improved. Finally, the results show that our method achieves 100\% success rate in dual-manipulator dual peg-in-hole tasks with a clearance of 0.2 mm, while robust to camera calibration errors.