Graph Repairs with Large Language Models: An Empirical Study

TL;DR

This paper empirically evaluates six open-source Large Language Models for automated repair of property graphs, highlighting their potential, limitations, and future research directions in improving graph data quality.

Contribution

It provides a comprehensive empirical assessment of LLMs for graph repair, comparing their accuracy, efficiency, and practical challenges in real-world scenarios.

Findings

LLMs can detect and correct graph errors with varying accuracy.

Repair quality and computational cost differ significantly among models.

The study identifies key limitations and future research directions for LLM-based graph repair.

Abstract

Property graphs are widely used in domains such as healthcare, finance, and social networks, but they often contain errors due to inconsistencies, missing data, or schema violations. Traditional rule-based and heuristic-driven graph repair methods are limited in their adaptability as they need to be tailored for each dataset. On the other hand, interactive human-in-the-loop approaches may become infeasible when dealing with large graphs, as the cost--both in terms of time and effort--of involving users becomes too high. Recent advancements in Large Language Models (LLMs) present new opportunities for automated graph repair by leveraging contextual reasoning and their access to real-world knowledge. We evaluate the effectiveness of six open-source LLMs in repairing property graphs. We assess repair quality, computational cost, and model-specific performance. Our experiments show that LLMs have the potential to detect and correct errors, with varying degrees of accuracy and efficiency. We discuss the strengths, limitations, and challenges of LLM-driven graph repair and outline future research directions for improving scalability and interpretability.