Scene Graph Modification as Incremental Structure Expanding
Xuming Hu, Zhijiang Guo, Yu Fu, Lijie Wen, Philip S. Yu

TL;DR
This paper introduces an incremental approach to scene graph modification that updates existing graphs based on natural language queries, improving accuracy and efficiency over previous methods.
Contribution
The paper proposes a novel incremental structure expanding method for scene graph modification, avoiding complete rebuilding and enhancing prediction accuracy.
Findings
Outperforms previous state-of-the-art models on four benchmarks.
Constructs a challenging new dataset with complex queries and larger graphs.
Demonstrates the effectiveness of incremental expansion in scene graph modification.
Abstract
A scene graph is a semantic representation that expresses the objects, attributes, and relationships between objects in a scene. Scene graphs play an important role in many cross modality tasks, as they are able to capture the interactions between images and texts. In this paper, we focus on scene graph modification (SGM), where the system is required to learn how to update an existing scene graph based on a natural language query. Unlike previous approaches that rebuilt the entire scene graph, we frame SGM as a graph expansion task by introducing the incremental structure expanding (ISE). ISE constructs the target graph by incrementally expanding the source graph without changing the unmodified structure. Based on ISE, we further propose a model that iterates between nodes prediction and edges prediction, inferring more accurate and harmonious expansion decisions progressively. In…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsMultimodal Machine Learning Applications · Advanced Graph Neural Networks · Topic Modeling
