3D Vision-guided Pick-and-Place Using Kuka LBR iiwa Robot

TL;DR

This paper develops a vision-guided pick-and-place system for a Kuka LBR iiwa robot using 3D cameras, enabling quick adaptation to new objects with minimal registration, tested across various hardware configurations.

Contribution

The paper introduces a flexible control system integrating calibration, pose registration, and object alignment for efficient pick-and-place tasks with minimal re-registration.

Findings

System successfully performs pick-and-place with limited re-registration

Compatible with multiple 3D cameras and robotic arms

Demonstrates quick adaptation to new objects

Abstract

This paper presents the development of a control system for vision-guided pick-and-place tasks using a robot arm equipped with a 3D camera. The main steps include camera intrinsic and extrinsic calibration, hand-eye calibration, initial object pose registration, objects pose alignment algorithm, and pick-and-place execution. The proposed system allows the robot be able to to pick and place object with limited times of registering a new object and the developed software can be applied for new object scenario quickly. The integrated system was tested using the hardware combination of kuka iiwa, Robotiq grippers (two finger gripper and three finger gripper) and 3D cameras (Intel realsense D415 camera, Intel realsense D435 camera, Microsoft Kinect V2). The whole system can also be modified for the combination of other robotic arm, gripper and 3D camera.