Multiple Heads are Better than One: Mixture of Modality Knowledge Experts for Entity Representation Learning

TL;DR

This paper introduces MoMoK, a novel framework that uses relation-guided modality experts to learn adaptive multi-modal entity representations, significantly improving knowledge graph completion tasks.

Contribution

It proposes relation-guided modality knowledge experts and a disentanglement strategy to better utilize multi-perspective features in multi-modal knowledge graphs.

Findings

MoMoK outperforms existing methods on four benchmarks.

The relation-guided experts effectively capture relation-aware features.

Disentangling experts improves representation quality.

Abstract

Learning high-quality multi-modal entity representations is an important goal of multi-modal knowledge graph (MMKG) representation learning, which can enhance reasoning tasks within the MMKGs, such as MMKG completion (MMKGC). The main challenge is to collaboratively model the structural information concealed in massive triples and the multi-modal features of the entities. Existing methods focus on crafting elegant entity-wise multi-modal fusion strategies, yet they overlook the utilization of multi-perspective features concealed within the modalities under diverse relational contexts. To address this issue, we introduce a novel framework with Mixture of Modality Knowledge experts (MoMoK for short) to learn adaptive multi-modal entity representations for better MMKGC. We design relation-guided modality knowledge experts to acquire relation-aware modality embeddings and integrate the predictions from multi-modalities to achieve joint decisions. Additionally, we disentangle the experts by minimizing their mutual information. Experiments on four public MMKG benchmarks demonstrate the outstanding performance of MoMoK under complex scenarios.