Grounding Scene Graphs on Natural Images via Visio-Lingual Message Passing
Aditay Tripathi, Anand Mishra, Anirban Chakraborty

TL;DR
This paper introduces VL-MPAG Net, a graph neural network that leverages scene graphs and message passing to improve object grounding in natural images, outperforming existing methods on multiple datasets.
Contribution
The paper proposes a novel visio-lingual message passing graph neural network for scene graph grounding, integrating semantic relationships to enhance object localization.
Findings
Significant performance improvements over baselines on four datasets.
Effective use of scene graph relationships for contextual object grounding.
Robustness across varied scene complexities.
Abstract
This paper presents a framework for jointly grounding objects that follow certain semantic relationship constraints given in a scene graph. A typical natural scene contains several objects, often exhibiting visual relationships of varied complexities between them. These inter-object relationships provide strong contextual cues toward improving grounding performance compared to a traditional object query-only-based localization task. A scene graph is an efficient and structured way to represent all the objects and their semantic relationships in the image. In an attempt towards bridging these two modalities representing scenes and utilizing contextual information for improving object localization, we rigorously study the problem of grounding scene graphs on natural images. To this end, we propose a novel graph neural network-based approach referred to as Visio-Lingual Message PAssing…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
Grounding Scene Graphs on Natural Images via Visio-Lingual Message Passing· youtube
Taxonomy
TopicsMultimodal Machine Learning Applications · Advanced Image and Video Retrieval Techniques · Advanced Neural Network Applications
MethodsGraph Neural Network
