ReInform: Selecting paths with reinforcement learning for contextualized link prediction

TL;DR

This paper introduces ReInform, a reinforcement learning approach that selects informative paths to enhance transformer-based link prediction models, outperforming previous RL answer search methods and improving accuracy by up to 13.5% MRR.

Contribution

ReInform is the first method to use RL for selecting paths to improve contextualized link prediction models, demonstrating significant performance gains.

Findings

ReInform outperforms RL-based answer search in link prediction tasks.

Combining RL with link prediction models yields up to 13.5% MRR improvement.

The approach is validated on WN18RR and FB15k-237 datasets.

Abstract

We propose to use reinforcement learning to inform transformer-based contextualized link prediction models by providing paths that are most useful for predicting the correct answer. This is in contrast to previous approaches, that either used reinforcement learning (RL) to directly search for the answer, or based their prediction on limited or randomly selected context. Our experiments on WN18RR and FB15k-237 show that contextualized link prediction models consistently outperform RL-based answer search, and that additional improvements (of up to 13.5% MRR) can be gained by combining RL with a link prediction model. The PyTorch implementation of the RL agent is available at https://github.com/marina-sp/reinform