Towards Parallel Computing on the Internet: Applications, Architectures, Models and Programming Tools

TL;DR

This survey reviews the development of parallel computing models, architectures, applications, and programming tools for Internet-based systems like computational grids, highlighting challenges like communication efficiency and system dynamism.

Contribution

It provides a comprehensive overview of the evolution of parallel computing on the Internet, emphasizing models, applications, and tools, and discusses open challenges such as dynamism and uncertainties.

Findings

Development of models emphasizing communication efficiency

Coverage of programming tools for computational grids

Identification of challenges like system dynamism

Abstract

The development of Internet wide resources for general purpose parallel computing poses the challenging task of matching computation and communication complexity. A number of parallel computing models exist that address this for traditional parallel architectures, and there are a number of emerging models that attempt to do this for large scale Internet-based systems like computational grids. In this survey we cover the three fundamental aspects -- application, architecture and model, and we show how they have been developed over the last decade. We also cover programming tools that are currently being used for parallel programming in computational grids. The trend in conventional computational models are to put emphasis on efficient communication between participating nodes by adapting different types of communication to network conditions. Effects of dynamism and uncertainties that arise in large scale systems are evidently important to understand and yet there is currently little work that addresses this from a parallel computing perspective.