Exploiting Edge-Oriented Reasoning for 3D Point-based Scene Graph Analysis

TL;DR

This paper introduces a novel 3D point-based scene graph generation framework that employs edge-oriented graph convolutional networks to improve scene understanding by explicitly modeling relationships and reasoning over 3D scenes.

Contribution

The paper proposes an edge-oriented GCN with twinning interaction mechanisms for independent evolution of nodes and edges in 3D scene graph reasoning, advancing scene understanding techniques.

Findings

Effective scene graph construction and reasoning in 3D point-based scenes.

Improved performance on traditional graph learning benchmarks.

Promising results in scene understanding and graph inference tasks.

Abstract

Scene understanding is a critical problem in computer vision. In this paper, we propose a 3D point-based scene graph generation ($\mathbf{SGG_{point}}$) framework to effectively bridge perception and reasoning to achieve scene understanding via three sequential stages, namely scene graph construction, reasoning, and inference. Within the reasoning stage, an EDGE-oriented Graph Convolutional Network ($\texttt{EdgeGCN}$) is created to exploit multi-dimensional edge features for explicit relationship modeling, together with the exploration of two associated twinning interaction mechanisms between nodes and edges for the independent evolution of scene graph representations. Overall, our integrated $\mathbf{SGG_{point}}$ framework is established to seek and infer scene structures of interest from both real-world and synthetic 3D point-based scenes. Our experimental results show promising edge-oriented reasoning effects on scene graph generation studies. We also demonstrate our method advantage on several traditional graph representation learning benchmark datasets, including the node-wise classification on citation networks and whole-graph recognition problems for molecular analysis.