A Light-Weight Communication Library for Distributed Computing

TL;DR

This paper introduces MPWide, a lightweight, platform-independent communication library optimized for long-distance message passing in distributed computing, demonstrated through large-scale cosmological simulations.

Contribution

The paper presents MPWide, a novel lightweight communication library that enables efficient message passing across distributed systems without administrative privileges.

Findings

Effective long-distance communication demonstrated in cosmological simulations

Library is platform independent and easy to install

Performance optimized for wide area networks

Abstract

We present MPWide, a platform independent communication library for performing message passing between computers. Our library allows coupling of several local MPI applications through a long distance network and is specifically optimized for such communications. The implementation is deliberately kept light-weight, platform independent and the library can be installed and used without administrative privileges. The only requirements are a C++ compiler and at least one open port to a wide area network on each site. In this paper we present the library, describe the user interface, present performance tests and apply MPWide in a large scale cosmological N-body simulation on a network of two computers, one in Amsterdam and the other in Tokyo.