AdaGrid: Adaptive Grid Search for Link Prediction Training Objective

TL;DR

AdaGrid is an adaptive, scalable grid search method that dynamically optimizes training hyperparameters for link prediction in graph neural networks, improving performance and efficiency over traditional methods.

Contribution

It introduces AdaGrid, a novel adaptive grid search algorithm that adjusts training hyperparameters during GNN training, reducing time and computational costs.

Findings

Boosts model performance up to 1.9%

Nine times more time-efficient than complete grid search

Model-agnostic and scalable approach

Abstract

One of the most important factors that contribute to the success of a machine learning model is a good training objective. Training objective crucially influences the model's performance and generalization capabilities. This paper specifically focuses on graph neural network training objective for link prediction, which has not been explored in the existing literature. Here, the training objective includes, among others, a negative sampling strategy, and various hyperparameters, such as edge message ratio which controls how training edges are used. Commonly, these hyperparameters are fine-tuned by complete grid search, which is very time-consuming and model-dependent. To mitigate these limitations, we propose Adaptive Grid Search (AdaGrid), which dynamically adjusts the edge message ratio during training. It is model agnostic and highly scalable with a fully customizable computational budget. Through extensive experiments, we show that AdaGrid can boost the performance of the models up to $1.9\%$ while being nine times more time-efficient than a complete search. Overall, AdaGrid represents an effective automated algorithm for designing machine learning training objectives.