LSMGraph: A High-Performance Dynamic Graph Storage System with Multi-Level CSR

TL;DR

LSMGraph is a disk-based dynamic graph storage system that combines LSM-trees and CSR to optimize both read and write performance, effectively handling large graph data with concurrent updates and analysis.

Contribution

It introduces a novel multi-level structure integrating LSM-trees and CSR, along with a vertex-grained version control for efficient concurrent operations.

Findings

Outperforms existing systems in update workloads

Achieves faster read performance with multi-level CSR

Maintains correctness during concurrent read/write operations

Abstract

The growing volume of graph data may exhaust the main memory. It is crucial to design a disk-based graph storage system to ingest updates and analyze graphs efficiently. However, existing dynamic graph storage systems suffer from read or write amplification and face the challenge of optimizing both read and write performance simultaneously. To address this challenge, we propose LSMGraph, a novel dynamic graph storage system that combines the write-friendly LSM-tree and the read-friendly CSR. It leverages the multi-level structure of LSM-trees to optimize write performance while utilizing the compact CSR structures embedded in the LSM-trees to boost read performance. LSMGraph uses a new memory structure, MemGraph, to efficiently cache graph updates and uses a multi-level index to speed up reads within the multi-level structure. Furthermore, LSMGraph incorporates a vertex-grained version control mechanism to mitigate the impact of LSM-tree compaction on read performance and ensure the correctness of concurrent read and write operations. Our evaluation shows that LSMGraph significantly outperforms state-of-the-art (graph) storage systems on both graph update and graph analytical workloads.