Asynchronous Distributed-Memory Triangle Counting and LCC with RMA Caching

TL;DR

This paper presents an asynchronous distributed-memory algorithm for triangle counting and local clustering coefficient computation using RMA caching, significantly improving scalability and performance over existing solutions.

Contribution

It introduces a fully asynchronous RMA-based approach with caching enhancements and application-specific scoring, enabling efficient large-scale graph analysis.

Findings

Achieves 14x speedup on distributed memory for LiveJournal1 graph.

Reduces total running time by up to 73% with caching.

Up to 100x faster than the TriC algorithm on scale-free graphs.

Abstract

Triangle count and local clustering coefficient are two core metrics for graph analysis. They find broad application in analyses such as community detection and link recommendation. Current state-of-the-art solutions suffer from synchronization overheads or expensive pre-computations needed to distribute the graph, achieving limited scaling capabilities. We propose a fully asynchronous implementation for triangle counting and local clustering coefficient based on 1D partitioning, using remote memory accesses for transferring data and avoid synchronization. Additionally, we show how these algorithms present data reuse on remote memory accesses and how the overall communication time can be improved by caching these accesses. Finally, we extend CLaMPI, a software-layer caching system for MPI RMA, to include application-specific scores for cached entries and influence the eviction procedure to improve caching efficiency. Our results show improvements on shared memory, and we achieve 14x speedup from 4 to 64 nodes for the LiveJournal1 graph on distributed memory. Moreover, we demonstrate how caching remote accesses reduces total running time by up to 73% with respect to a non-cached version. Finally, we compare our implementation to TriC, the 2020 graph champion paper, and achieve up to 100x faster results for scale-free graphs.