Atrapos: Real-time Evaluation of Metapath Query Workloads

TL;DR

ATRAPOS is a novel system that enables real-time evaluation of metapath queries on large heterogeneous information networks by leveraging caching and efficient computation, significantly improving performance.

Contribution

The paper introduces ATRAPOS, a new approach that combines sparse matrix multiplication with real-time caching of intermediate results using a novel data structure, the Overlap Tree.

Findings

ATRAPOS outperforms existing methods in all tested scenarios.

It accelerates exploratory analysis on large HINs.

The approach effectively detects and reuses frequent sub-metapaths.

Abstract

Heterogeneous information networks (HINs) represent different types of entities and relationships between them. Exploring, analysing, and extracting knowledge from such networks relies on metapath queries that identify pairs of entities connected by relationships of diverse semantics. While the real-time evaluation of metapath query workloads on large, web-scale HINs is highly demanding in computational cost, current approaches do not exploit interrelationships among the queries. In this paper, we present ATRAPOS, a new approach for the real-time evaluation of metapath query workloads that leverages a combination of efficient sparse matrix multiplication and intermediate result caching. ATRAPOS selects intermediate results to cache and reuse by detecting frequent sub-metapaths among workload queries in real time, using a tailor-made data structure, the Overlap Tree, and an associated caching policy. Our experimental study on real data shows that ATRAPOS accelerates exploratory data analysis and mining on HINs, outperforming off-the-shelf caching approaches and state-of-the-art research prototypes in all examined scenarios. -- Note that this version of our work is more extended than the one presented in TheWebConf 2023 (doi: 10.1145/3543507.3583322)