Affordable HPC: Leveraging Small Clusters for Big Data and Graph Computing

TL;DR

This paper investigates cost-effective strategies for academic researchers to build small, efficient computing clusters, addressing hardware selection, performance optimization, and network analysis to enable affordable high-performance computing.

Contribution

It introduces methods for hardware optimization, bandwidth mitigation techniques, and a GNN framework for analyzing parallelism in small clusters, advancing affordable HPC solutions.

Findings

Cost-effective cluster assembly strategies identified

Bandwidth mitigation techniques improve GPU performance

GNN framework effectively analyzes parallelism

Abstract

This study explores strategies for academic researchers to optimize computational resources within limited budgets, focusing on building small, efficient computing clusters. It delves into the comparative costs of purchasing versus renting servers, guided by market research and economic theories on tiered pricing. The paper offers detailed insights into the selection and assembly of hardware components such as CPUs, GPUs, and motherboards tailored to specific research needs. It introduces innovative methods to mitigate the performance issues caused by PCIe switch bandwidth limitations in order to enhance GPU task scheduling. Furthermore, a Graph Neural Network (GNN) framework is proposed to analyze and optimize parallelism in computing networks.