Multi-Modal Dataset Acquisition for Photometrically Challenging Object

TL;DR

This paper introduces a novel multi-modal dataset acquisition pipeline that improves the accuracy, realism, and coverage of 3D perception datasets for photometrically challenging objects, using robotic and freehand methods.

Contribution

It presents a new annotation and data collection pipeline combining robotic kinematics, infrared tracking, and calibration for high-quality 3D datasets.

Findings

Enhanced dataset accuracy and realism

Wider viewpoint coverage achieved with freehand procedure

Improved 3D object and camera pose annotations

Abstract

This paper addresses the limitations of current datasets for 3D vision tasks in terms of accuracy, size, realism, and suitable imaging modalities for photometrically challenging objects. We propose a novel annotation and acquisition pipeline that enhances existing 3D perception and 6D object pose datasets. Our approach integrates robotic forward-kinematics, external infrared trackers, and improved calibration and annotation procedures. We present a multi-modal sensor rig, mounted on a robotic end-effector, and demonstrate how it is integrated into the creation of highly accurate datasets. Additionally, we introduce a freehand procedure for wider viewpoint coverage. Both approaches yield high-quality 3D data with accurate object and camera pose annotations. Our methods overcome the limitations of existing datasets and provide valuable resources for 3D vision research.