Towards Neural Graph Data Management

TL;DR

This paper introduces NGDBench, a comprehensive benchmark for neural graph data management that evaluates models on complex graph queries across multiple domains, highlighting current limitations in structured reasoning and robustness.

Contribution

The paper presents NGDBench, the first benchmark supporting full Cypher queries with realistic noise and dynamic operations, enabling more effective evaluation of neural graph data systems.

Findings

State-of-the-art models show limited reasoning capabilities.

Models struggle with noise robustness in graph queries.

Current approaches lack precision in complex graph analysis.

Abstract

While AI systems have made remarkable progress in processing unstructured text, structured data such as graphs stored in databases, continues to grow rapidly yet remains difficult for neural models to effectively utilize. We introduce NGDBench, a unified benchmark for evaluating neural graph database capabilities across five diverse domains, including finance, medicine, and AI agent tooling. Unlike prior benchmarks limited to elementary logical operations, NGDBench supports the full Cypher query language, enabling complex pattern matching, variable-length paths, and numerical aggregations, while incorporating realistic noise injection and dynamic data management operations. Our evaluation of state-of-the-art LLMs and RAG methods reveals significant limitations in structured reasoning, noise robustness, and analytical precision, establishing NGDBench as a critical testbed for advancing neural graph data management. Our code and data are available at https://github.com/HKUST-KnowComp/NGDBench.