Exploring the Hierarchy in Relation Labels for Scene Graph Generation
Yi Zhou, Shuyang Sun, Chao Zhang, Yikang Li, Wanli Ouyang

TL;DR
This paper introduces a hierarchy-guided approach for scene graph generation that leverages predicate hierarchies to improve feature learning, significantly boosting performance over existing methods.
Contribution
It proposes the Hierarchy Guided Feature Learning and Hierarchy Guided Module to incorporate predicate hierarchies into scene graph generation, enhancing feature representation and accuracy.
Findings
Achieved up to 33% relative gain in Recall@50
Improved performance across multiple datasets
Effective utilization of predicate hierarchies
Abstract
By assigning each relationship a single label, current approaches formulate the relationship detection as a classification problem. Under this formulation, predicate categories are treated as completely different classes. However, different from the object labels where different classes have explicit boundaries, predicates usually have overlaps in their semantic meanings. For example, sit\_on and stand\_on have common meanings in vertical relationships but different details of how these two objects are vertically placed. In order to leverage the inherent structures of the predicate categories, we propose to first build the language hierarchy and then utilize the Hierarchy Guided Feature Learning (HGFL) strategy to learn better region features of both the coarse-grained level and the fine-grained level. Besides, we also propose the Hierarchy Guided Module (HGM) to utilize the…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsMultimodal Machine Learning Applications · Topic Modeling · Video Analysis and Summarization
