Efficient Semi-External Depth-First Search

TL;DR

This paper introduces EP-DFS, a semi-external DFS algorithm that efficiently handles large graphs by reducing I/O operations and CPU complexity, outperforming existing methods in experiments.

Contribution

The paper presents a novel semi-external DFS algorithm with a lightweight index, improving efficiency and simplicity over traditional approaches.

Findings

EP-DFS significantly reduces I/O operations.

EP-DFS outperforms existing algorithms in experiments.

The approach is implementation-friendly with less random I/O.

Abstract

As the sizes of graphs grow rapidly, currently many real-world graphs can hardly be loaded in the main memory. It becomes a hot topic to compute depth-first search (DFS) results, i.e., depth-first order or DFS-Tree, on semi-external memory model. Semi-external algorithms assume the main memory could at least hold a spanning tree T of a graph G, and gradually restructure T into a DFS-Tree, which is non-trivial. In this paper, we present a comprehensive study of semi-external DFS problem. Based on our theoretical analysis of its main challenge, we introduce a new semi-external DFS algorithm, named EP-DFS, with a lightweight index N+-index. Unlike traditional algorithms, we focus on addressing such complex problem efficiently not only with less I/Os, but also with simpler CPU calculation (implementation-friendly) and less random I/O accesses (key-to-efficiency). Extensive experimental evaluation is conducted on both synthetic and real graphs. The experimental results confirm that our EP-DFS algorithm significantly outperforms existing algorithms.