Relation Regularized Scene Graph Generation
Yuyu Guo, Lianli Gao, Jingkuan Song, Peng Wang, Nicu Sebe, Heng Tao, Shen, Xuelong Li

TL;DR
This paper introduces R2-Net, a relation regularized network that uses an affinity matrix and graph convolutional networks to improve scene graph generation by better encoding relationships between objects.
Contribution
The paper proposes a novel relation regularized network (R2-Net) that leverages an affinity matrix and GCNs to enhance object feature refinement and scene graph generation.
Findings
Significant performance improvements on Visual Genome dataset.
Effective relation encoding improves object label refinement.
Ablation studies confirm the importance of proposed components.
Abstract
Scene graph generation (SGG) is built on top of detected objects to predict object pairwise visual relations for describing the image content abstraction. Existing works have revealed that if the links between objects are given as prior knowledge, the performance of SGG is significantly improved. Inspired by this observation, in this article, we propose a relation regularized network (R2-Net), which can predict whether there is a relationship between two objects and encode this relation into object feature refinement and better SGG. Specifically, we first construct an affinity matrix among detected objects to represent the probability of a relationship between two objects. Graph convolution networks (GCNs) over this relation affinity matrix are then used as object encoders, producing relation-regularized representations of objects. With these relation-regularized features, our R2-Net…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsMultimodal Machine Learning Applications · Advanced Image and Video Retrieval Techniques · Image Retrieval and Classification Techniques
MethodsConvolution
