DRILL-- Deep Reinforcement Learning for Refinement Operators in $\mathcal{ALC}$

TL;DR

DRILL introduces a deep reinforcement learning method using convolutional deep Q-learning to efficiently guide class expression learning in $ ext{ALC}$, significantly speeding up convergence compared to existing approaches.

Contribution

This work presents DRILL, a novel deep reinforcement learning framework that improves the efficiency of class expression learning in $ ext{ALC}$ by predicting future rewards to guide search.

Findings

DRILL computes expected rewards for over 1000 class expressions per second.

DRILL converges at least 2.7 times faster than state-of-the-art methods on benchmarks.

Open-source implementation provided for reproducibility.

Abstract

Approaches based on refinement operators have been successfully applied to class expression learning on RDF knowledge graphs. These approaches often need to explore a large number of concepts to find adequate hypotheses. This need arguably stems from current approaches relying on myopic heuristic functions to guide their search through an infinite concept space. In turn, deep reinforcement learning provides effective means to address myopia by estimating how much discounted cumulated future reward states promise. In this work, we leverage deep reinforcement learning to accelerate the learning of concepts in $\mathcal{ALC}$ by proposing DRILL -- a novel class expression learning approach that uses a convolutional deep Q-learning model to steer its search. By virtue of its architecture, DRILL is able to compute the expected discounted cumulated future reward of more than $10^3$ class expressions in a second on standard hardware. We evaluate DRILL on four benchmark datasets against state-of-the-art approaches. Our results suggest that DRILL converges to goal states at least 2.7$\times$ faster than state-of-the-art models on all benchmark datasets. We provide an open-source implementation of our approach, including training and evaluation scripts as well as pre-trained models.