KITchen: A Real-World Benchmark and Dataset for 6D Object Pose Estimation in Kitchen Environments

TL;DR

KITchen introduces a comprehensive real-world dataset and benchmark for 6D object pose estimation in kitchen environments, addressing the gap between existing datasets and real-world robotic manipulation challenges.

Contribution

The paper presents a new dataset, benchmark, and semi-automated annotation pipeline tailored for 6D pose estimation in diverse kitchen settings, reflecting real-world robotic tasks.

Findings

Dataset includes 205k RGBD images of 111 objects in kitchens.

Semi-automated annotation pipeline reduces labeling effort.

Benchmark enables evaluation of pose estimation methods in realistic scenarios.

Abstract

Despite the recent progress on 6D object pose estimation methods for robotic grasping, a substantial performance gap persists between the capabilities of these methods on existing datasets and their efficacy in real-world grasping and mobile manipulation tasks, particularly when robots rely solely on their monocular egocentric field of view (FOV). Existing real-world datasets primarily focus on table-top grasping scenarios, where a robot arm is placed in a fixed position and the objects are centralized within the FOV of fixed external camera(s). Assessing performance on such datasets may not accurately reflect the challenges encountered in everyday grasping and mobile manipulation tasks within kitchen environments such as retrieving objects from higher shelves, sinks, dishwashers, ovens, refrigerators, or microwaves. To address this gap, we present KITchen, a novel benchmark designed specifically for estimating the 6D poses of objects located in diverse positions within kitchen settings. For this purpose, we recorded a comprehensive dataset comprising around 205k real-world RGBD images for 111 kitchen objects captured in two distinct kitchens, utilizing a humanoid robot with its egocentric perspectives. Subsequently, we developed a semi-automated annotation pipeline, to streamline the labeling process of such datasets, resulting in the generation of 2D object labels, 2D object segmentation masks, and 6D object poses with minimal human effort. The benchmark, the dataset, and the annotation pipeline will be publicly available at https://kitchen-dataset.github.io/KITchen.