Swift and Sure: Hardness-aware Contrastive Learning for Low-dimensional Knowledge Graph Embeddings

TL;DR

This paper introduces HaLE, a novel contrastive learning framework for low-dimensional knowledge graph embeddings that improves training efficiency and performance by incorporating hardness-aware mechanisms and query sampling.

Contribution

The paper proposes a hardness-aware contrastive learning framework for low-dimensional KGE that reduces training time and enhances model performance using a new loss function and activation mechanism.

Findings

HaLE improves training speed and performance on multiple datasets.

Models trained with HaLE are competitive with state-of-the-art methods.

Effective in both low- and high-dimensional settings.

Abstract

Knowledge graph embedding (KGE) has shown great potential in automatic knowledge graph (KG) completion and knowledge-driven tasks. However, recent KGE models suffer from high training cost and large storage space, thus limiting their practicality in real-world applications. To address this challenge, based on the latest findings in the field of Contrastive Learning, we propose a novel KGE training framework called Hardness-aware Low-dimensional Embedding (HaLE). Instead of the traditional Negative Sampling, we design a new loss function based on query sampling that can balance two important training targets, Alignment and Uniformity. Furthermore, we analyze the hardness-aware ability of recent low-dimensional hyperbolic models and propose a lightweight hardness-aware activation mechanism. The experimental results show that in the limited training time, HaLE can effectively improve the performance and training speed of KGE models on five commonly-used datasets. After training just a few minutes, the HaLE-trained models are competitive compared to the state-of-the-art models in both low- and high-dimensional conditions.