A mixed-reality dataset for category-level 6D pose and size estimation of hand-occluded containers

TL;DR

This paper introduces a large mixed-reality dataset of hand-occluded household containers to improve category-level 6D pose and size estimation, addressing challenges posed by intra-class variations and occlusions.

Contribution

The paper presents a new mixed-reality dataset with 138,240 images of hand-held containers for training and testing pose estimation models, enhancing robustness to occlusions and variations.

Findings

Improved 6D pose estimation accuracy using the dataset

Demonstrated the dataset's effectiveness in handling hand occlusions

Provided insights into dataset impact on model performance

Abstract

Estimating the 6D pose and size of household containers is challenging due to large intra-class variations in the object properties, such as shape, size, appearance, and transparency. The task is made more difficult when these objects are held and manipulated by a person due to varying degrees of hand occlusions caused by the type of grasps and by the viewpoint of the camera observing the person holding the object. In this paper, we present a mixed-reality dataset of hand-occluded containers for category-level 6D object pose and size estimation. The dataset consists of 138,240 images of rendered hands and forearms holding 48 synthetic objects, split into 3 grasp categories over 30 real backgrounds. We re-train and test an existing model for 6D object pose estimation on our mixed-reality dataset. We discuss the impact of the use of this dataset in improving the task of 6D pose and size estimation.