Maintaining Leiden Communities in Large Dynamic Graphs

TL;DR

This paper introduces HIT-Leiden, a novel algorithm for efficiently maintaining high-quality Leiden communities in large, rapidly evolving dynamic graphs, significantly improving update speed while preserving community quality.

Contribution

The paper presents HIT-Leiden, an innovative incremental algorithm that reduces affected vertices and maintains community quality, outperforming existing methods in speed and scalability.

Findings

HIT-Leiden achieves up to five orders of magnitude speedup over existing solutions.

The algorithm maintains community quality comparable to state-of-the-art methods.

HIT-Leiden meets strict latency requirements in production environments.

Abstract

Community detection is a foundational capability in large-scale industrial graph analytics, powering applications such as fraud-ring discovery, recommendation systems, and hierarchical indexing for retrieval-augmented generation. Among modularity-based methods, the Leiden algorithm has been widely adopted in production because it delivers high-quality communities with connectivity guarantees. However, real-world graphs evolve continuously, and timely community updates are needed to keep downstream features and retrieval indices fresh. Meanwhile, existing dynamic Leiden approaches recompute the communities whenever their vertices and edges change, thereby almost degrading to near-full recomputation under frequent updates. To alleviate the efficiency issue, we study the efficient maintenance of Leiden communities in large dynamic graphs and present a novel algorithm, called Hierarchical Incremental Tree Leiden (HIT-Leiden). We first provide a boundedness analysis showing that prior incremental Leiden methods can incur essentially unbounded work even for small updates. Guided by this analysis, we propose HIT-Leiden which effectively reduces the range of affected vertices by maintaining connected components and hierarchical community structures. Extensive experiments on large real-world dynamic graphs demonstrate that HIT-Leiden achieves community quality comparable to the state-of-the-art competitors while delivering speedups of up to five orders of magnitude over existing solutions. The production deployment results show that HIT-Leiden meets stringent latency requirements under high-rate updates at scale.