Automatic Construction of Real-World Datasets for 3D Object Localization using Two Cameras

TL;DR

This paper presents a method to automatically generate large, real-world stereo image datasets with accurate 3D object position labels using an industrial robot, facilitating training of end-to-end localization models.

Contribution

It introduces a novel approach to create labeled datasets for 3D object localization using robot geometry, eliminating manual annotation.

Findings

Successfully generated a screw-driver localization dataset with stereo images.

Enabled training of CNN regressors for end-to-end stereo localization.

Demonstrated the effectiveness of robot-assisted dataset creation for 3D localization.

Abstract

Unlike classification, position labels cannot be assigned manually by humans. For this reason, generating supervision for precise object localization is a hard task. This paper details a method to create large datasets for 3D object localization, with real world images, using an industrial robot to generate position labels. By knowledge of the geometry of the robot, we are able to automatically synchronize the images of the two cameras and the object 3D position. We applied it to generate a screw-driver localization dataset with stereo images, using a KUKA LBR iiwa robot. This dataset could then be used to train a CNN regressor to learn end-to-end stereo object localization from a set of two standard uncalibrated cameras.