Extending TWIG: Zero-Shot Predictive Hyperparameter Selection for KGEs based on Graph Structure

TL;DR

This paper extends the TWIG model to predict KGE performance in zero-shot settings, enabling hyperparameter selection for knowledge graph embeddings based solely on graph structure, reducing the need for extensive hyperparameter tuning.

Contribution

The work demonstrates that TWIG can effectively predict KGE performance across different KGs and hyperparameters, advancing zero-shot hyperparameter optimization methods.

Findings

TWIG summarizes KGE performance across various hyperparameters and KGs.

TWIG can predict hyperparameter performance on unseen KGs in zero-shot settings.

Potential for pre-hoc hyperparameter selection without full search.

Abstract

Knowledge Graphs (KGs) have seen increasing use across various domains -- from biomedicine and linguistics to general knowledge modelling. In order to facilitate the analysis of knowledge graphs, Knowledge Graph Embeddings (KGEs) have been developed to automatically analyse KGs and predict new facts based on the information in a KG, a task called "link prediction". Many existing studies have documented that the structure of a KG, KGE model components, and KGE hyperparameters can significantly change how well KGEs perform and what relationships they are able to learn. Recently, the Topologically-Weighted Intelligence Generation (TWIG) model has been proposed as a solution to modelling how each of these elements relate. In this work, we extend the previous research on TWIG and evaluate its ability to simulate the output of the KGE model ComplEx in the cross-KG setting. Our results are twofold. First, TWIG is able to summarise KGE performance on a wide range of hyperparameter settings and KGs being learned, suggesting that it represents a general knowledge of how to predict KGE performance from KG structure. Second, we show that TWIG can successfully predict hyperparameter performance on unseen KGs in the zero-shot setting. This second observation leads us to propose that, with additional research, optimal hyperparameter selection for KGE models could be determined in a pre-hoc manner using TWIG-like methods, rather than by using a full hyperparameter search.