DUQIM-Net: Probabilistic Object Hierarchy Representation for Multi-View Manipulation

TL;DR

DUQIM-Net is a probabilistic hierarchical object representation method that improves multi-view manipulation in cluttered scenes by inferring object relationships and aiding grasping tasks.

Contribution

The paper introduces DUQIM-Net, which uses a novel adjacency head in a Transformer-based model to infer object hierarchies for better manipulation in cluttered environments.

Findings

Adj-Net surpasses state-of-the-art in relationship inference on VMRD.

DUQIM-Net outperforms existing methods in bin clearing tasks.

Probabilistic hierarchy modeling enhances manipulation success.

Abstract

Object manipulation in cluttered scenes is a difficult and important problem in robotics. To efficiently manipulate objects, it is crucial to understand their surroundings, especially in cases where multiple objects are stacked one on top of the other, preventing effective grasping. We here present DUQIM-Net, a decision-making approach for object manipulation in a setting of stacked objects. In DUQIM-Net, the hierarchical stacking relationship is assessed using Adj-Net, a model that leverages existing Transformer Encoder-Decoder object detectors by adding an adjacency head. The output of this head probabilistically infers the underlying hierarchical structure of the objects in the scene. We utilize the properties of the adjacency matrix in DUQIM-Net to perform decision making and assist with object-grasping tasks. Our experimental results show that Adj-Net surpasses the state-of-the-art in object-relationship inference on the Visual Manipulation Relationship Dataset (VMRD), and that DUQIM-Net outperforms comparable approaches in bin clearing tasks.