Scalable Unix Commands for Parallel Processors: A High-Performance Implementation

TL;DR

This paper introduces a set of parallelized Unix commands designed for high-performance computing environments, demonstrating their implementation and performance on a 256-node Linux cluster.

Contribution

It presents a new family of MPI-based parallel Unix commands, expanding traditional command functionality for large-scale parallel processing.

Findings

Achieved efficient performance on 256-node Linux cluster

Developed open-source MPI applications for Unix commands

Enhanced command capabilities for parallel processing environments

Abstract

We describe a family of MPI applications we call the Parallel Unix Commands. These commands are natural parallel versions of common Unix user commands such as ls, ps, and find, together with a few similar commands particular to the parallel environment. We describe the design and implementation of these programs and present some performance results on a 256-node Linux cluster. The Parallel Unix Commands are open source and freely available.