Croppable Knowledge Graph Embedding

TL;DR

This paper introduces MED, a novel training framework for knowledge graph embeddings that enables creating multiple sub-models of different dimensions from a single training, enhancing efficiency and flexibility in AI applications.

Contribution

MED allows for a single training process to produce croppable sub-models of various dimensions, reducing costs and increasing adaptability in knowledge graph embedding tasks.

Findings

MED improves low-dimensional sub-model performance.

High-dimensional sub-models retain low-dimensional capacity.

Framework demonstrates effectiveness across multiple datasets and models.

Abstract

Knowledge Graph Embedding (KGE) is a common approach for Knowledge Graphs (KGs) in AI tasks. Embedding dimensions depend on application scenarios. Requiring a new dimension means training a new KGE model from scratch, increasing cost and limiting efficiency and flexibility. In this work, we propose a novel KGE training framework MED. It allows one training to obtain a croppable KGE model for multiple scenarios with different dimensional needs. Sub-models of required dimensions can be directly cropped and used without extra training. In MED, we propose a mutual learning mechanism to improve the low-dimensional sub-models and make high-dimensional sub-models retain the low-dimensional sub-models' capacity, an evolutionary improvement mechanism to promote the high-dimensional sub-models to master the triple that the low-dimensional sub-models can not, and a dynamic loss weight to adaptively balance the multiple losses. Experiments on 4 KGE models across 4 standard KG completion datasets, 3 real-world scenarios using a large-scale KG, and extending MED to the BERT language model demonstrate its effectiveness, high efficiency, and flexible extensibility.