Counting Still Counts: Understanding Neural Complex Query Answering Through Query Relaxation

TL;DR

This paper critically evaluates neural models for complex query answering over knowledge graphs, revealing they do not consistently outperform simple query relaxation methods and highlighting the need for stronger baselines.

Contribution

It provides a systematic comparison between neural CQA models and a training-free query relaxation approach, challenging assumptions about neural models' reasoning capabilities.

Findings

Neural and relaxation-based methods perform similarly across datasets.

Their answer sets have little overlap, but combining them improves results.

Current neural models do not consistently outperform query relaxation strategies.

Abstract

Neural methods for Complex Query Answering (CQA) over knowledge graphs (KGs) are widely believed to learn patterns that generalize beyond explicit graph structure, allowing them to infer answers that are unreachable through symbolic query processing. In this work, we critically examine this assumption through a systematic analysis comparing neural CQA models with an alternative, training-free query relaxation strategy that retrieves possible answers by relaxing query constraints and counting resulting paths. Across multiple datasets and query structures, we find several cases where neural and relaxation-based approaches perform similarly, with no neural model consistently outperforming the latter. Moreover, a similarity analysis reveals that their retrieved answers exhibit little overlap, and that combining their outputs consistently improves performance. These results call for a re-evaluation of progress in neural query answering: despite their complexity, current models fail to subsume the reasoning patterns captured by query relaxation. Our findings highlight the importance of stronger non-neural baselines and suggest that future neural approaches could benefit from incorporating principles of query relaxation.