DSparsE: Dynamic Sparse Embedding for Knowledge Graph Completion
Chuhong Yang, Bin Li, Nan Wu
TL;DR
This paper introduces DSparsE, a dynamic sparse embedding model for knowledge graph completion that improves robustness and reduces overfitting, achieving state-of-the-art results on multiple datasets.
Contribution
The paper proposes a novel dynamic sparse embedding approach with a shallow encoder and residual deep decoder, replacing dense layers with sparse connections to enhance performance.
Findings
Achieves state-of-the-art Hits@1 on FB15k-237, WN18RR, YAGO3-10 datasets.
Demonstrates robustness and reduced overfitting through sparse connection layers.
Ablation study confirms the effectiveness of dynamic and relation-aware layers.
Abstract
Addressing the incompleteness problem in knowledge graph remains a significant challenge. Current knowledge graph completion methods have their limitations. For example, ComDensE is prone to overfitting and suffers from the degradation with the increase of network depth while InteractE has the limitations in feature interaction and interpretability. To this end, we propose a new method called dynamic sparse embedding (DSparsE) for knowledge graph completion. The proposed model embeds the input entity-relation pairs by a shallow encoder composed of a dynamic layer and a relation-aware layer. Subsequently, the concatenated output of the dynamic layer and relation-aware layer is passed through a projection layer and a deep decoder with residual connection structure. This model ensures the network robustness and maintains the capability of feature extraction. Furthermore, the conventional…
Peer Reviews
Decision·Submitted to ICLR 2024
- The proposed architecture utilizes various methods to prevent some of the well known problems, especially the overfitting problem which is important for knowledge graph completion. - The paper provides ablation studies to show the benefit of implementing proposed methods.
- Better explanations can be included. For instance, the paper states the expert kernels, but does not explicitly explain what is the expert kernels or why it is "expert". - Better figures and figure captions can be written. The architecture figures are small and the explanations in the captions is very limited. - Some statistical testings (or critical plots) for the comparing methods would make the arguments of the paper stronger. - Comparison of computational time would be necessary to address
1. Authors tried to investigate how powerful the MLP is for KGC task by developing model based on pure MLP layers. This is significantly different to existing methods and very interesting. 2. Though the experiment results of DSparsE is not comparable to state-of-the-art. But it shows DSparsE is effective for KGC tasks.
1. Though the overall model architecture is novel, the key advantage of pure MLP-based model such as DSparsE is unclear. As I understand, it is not efficiency since 15(35) experts are set for FB15k-237(WN18RR) with each expert represented by an MLP layer, which will introduce a lot extra parameters compared to 1 expert. It is also not superior performance, since the link prediction results of DSparsE is comparable to existing methods, such as RESCAL and ComDensE. 2. How do the different modules
1. The paper is clearly written and easy to follow. 2. This model combines dynamic layer and residual structure, wwhich enables neural networks to better perform information fusion
1. The number of datasets is only 2, relatively limited. 2. The supplementary experiments in this article are not sufficient. For instance, Figure 4 provides a comparison between DSparseE and one baseline on a single dataset. Both the number of baselines for comparison and the diversity of datasets should be increased. Additionally, Figure 5 only illustrates three scenarios of expert quantity, making it challenging to discern a clear and definitive trend. In Figure 6, the blue and orange lines a
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsAdvanced Graph Neural Networks · Cognitive Computing and Networks · Semantic Web and Ontologies
MethodsResidual Connection