FAMST: Fast Approximate Minimum Spanning Tree Construction for Large-Scale and High-Dimensional Data

TL;DR

FAMST is a fast, approximate algorithm for constructing minimum spanning trees on large, high-dimensional datasets, significantly reducing computational time while maintaining low approximation error.

Contribution

The paper introduces FAMST, a novel three-phase algorithm that efficiently constructs approximate MSTs with theoretical guarantees and practical scalability for large-scale, high-dimensional data.

Findings

Achieves $ ext{O}(dn ext{log} n)$ time complexity.

Provides speedups of up to 1000x over exact algorithms.

Enables MST analysis on datasets with millions of points.

Abstract

We present Fast Approximate Minimum Spanning Tree (FAMST), a novel algorithm that addresses the computational challenges of constructing Minimum Spanning Trees (MSTs) for large-scale and high-dimensional datasets. FAMST utilizes a three-phase approach: Approximate Nearest Neighbor (ANN) graph construction, ANN inter-component connection, and iterative edge refinement. For a dataset of $n$ points in a $d$-dimensional space, FAMST achieves $\mathcal{O}(dn \log n)$ time complexity and $\mathcal{O}(dn + kn)$ space complexity when $k$ nearest neighbors are considered, which is a significant improvement over the $\mathcal{O}(n^2)$ time and space complexity of traditional methods. Experiments across diverse datasets demonstrate that FAMST achieves remarkably low approximation errors while providing speedups of up to 1000$\times$ compared to exact MST algorithms. We analyze how the key hyperparameters, $k$ (neighborhood size) and $\lambda$ (inter-component edges), affect performance, providing practical guidelines for hyperparameter selection. FAMST enables MST-based analysis on datasets with millions of points and thousands of dimensions, extending the applicability of MST techniques to problem scales previously considered infeasible.