Modality-Aware Negative Sampling for Multi-modal Knowledge Graph Embedding

TL;DR

This paper introduces Modality-Aware Negative Sampling (MANS), a novel, efficient method for improving multi-modal knowledge graph embedding by aligning structural and visual information, outperforming existing approaches.

Contribution

The paper proposes MANS, a lightweight negative sampling method that effectively incorporates multi-modal data into knowledge graph embeddings, addressing limitations of previous methods.

Findings

MANS outperforms existing negative sampling methods on benchmark datasets.

MANS effectively aligns structural and visual embeddings for better multi-modal KGE.

Empirical results confirm the efficiency and effectiveness of MANS.

Abstract

Negative sampling (NS) is widely used in knowledge graph embedding (KGE), which aims to generate negative triples to make a positive-negative contrast during training. However, existing NS methods are unsuitable when multi-modal information is considered in KGE models. They are also inefficient due to their complex design. In this paper, we propose Modality-Aware Negative Sampling (MANS) for multi-modal knowledge graph embedding (MMKGE) to address the mentioned problems. MANS could align structural and visual embeddings for entities in KGs and learn meaningful embeddings to perform better in multi-modal KGE while keeping lightweight and efficient. Empirical results on two benchmarks demonstrate that MANS outperforms existing NS methods. Meanwhile, we make further explorations about MANS to confirm its effectiveness.