CascadER: Cross-Modal Cascading for Knowledge Graph Link Prediction

TL;DR

CascadER is a novel cross-modal cascading architecture that enhances knowledge graph link prediction accuracy and efficiency by selectively reranking base models with language models, setting new state-of-the-art results.

Contribution

The paper introduces CascadER, a tiered ranking system that combines knowledge graph embeddings and language models efficiently for improved link prediction.

Findings

Improves MRR by up to 9 points over KGE baselines.

Sets new state-of-the-art on four benchmarks.

Achieves efficiency gains of one or more orders of magnitude.

Abstract

Knowledge graph (KG) link prediction is a fundamental task in artificial intelligence, with applications in natural language processing, information retrieval, and biomedicine. Recently, promising results have been achieved by leveraging cross-modal information in KGs, using ensembles that combine knowledge graph embeddings (KGEs) and contextual language models (LMs). However, existing ensembles are either (1) not consistently effective in terms of ranking accuracy gains or (2) impractically inefficient on larger datasets due to the combinatorial explosion problem of pairwise ranking with deep language models. In this paper, we propose a novel tiered ranking architecture CascadER to maintain the ranking accuracy of full ensembling while improving efficiency considerably. CascadER uses LMs to rerank the outputs of more efficient base KGEs, relying on an adaptive subset selection scheme aimed at invoking the LMs minimally while maximizing accuracy gain over the KGE. Extensive experiments demonstrate that CascadER improves MRR by up to 9 points over KGE baselines, setting new state-of-the-art performance on four benchmarks while improving efficiency by one or more orders of magnitude over competitive cross-modal baselines. Our empirical analyses reveal that diversity of models across modalities and preservation of individual models' confidence signals help explain the effectiveness of CascadER, and suggest promising directions for cross-modal cascaded architectures. Code and pretrained models are available at https://github.com/tsafavi/cascader.