Explore Contextual Information for 3D Scene Graph Generation

TL;DR

This paper introduces a new framework for 3D scene graph generation that leverages contextual information to improve fine-grained classification, multi-label relation prediction, and overall accuracy.

Contribution

It proposes a novel approach with graph feature extraction and hierarchical reasoning modules to enhance 3D SGG performance, especially in relationship prediction.

Findings

Achieves superior performance on 3DSSG dataset

Improves accuracy in multi-label relation prediction

Enhances fine-grained entity classification

Abstract

3D scene graph generation (SGG) has been of high interest in computer vision. Although the accuracy of 3D SGG on coarse classification and single relation label has been gradually improved, the performance of existing works is still far from being perfect for fine-grained and multi-label situations. In this paper, we propose a framework fully exploring contextual information for the 3D SGG task, which attempts to satisfy the requirements of fine-grained entity class, multiple relation labels, and high accuracy simultaneously. Our proposed approach is composed of a Graph Feature Extraction module and a Graph Contextual Reasoning module, achieving appropriate information-redundancy feature extraction, structured organization, and hierarchical inferring. Our approach achieves superior or competitive performance over previous methods on the 3DSSG dataset, especially on the relationship prediction sub-task.