Re-evaluating scaling methods for distributed parallel systems

TL;DR

This paper clarifies the correct interpretation of Amdahl's Law for distributed systems, emphasizing its fundamental role in understanding parallel processing limits and addressing common misconceptions.

Contribution

It provides a comprehensive model and analysis to correctly interpret Amdahl's Law in distributed systems, resolving debates and clarifying its universal applicability.

Findings

Amdahl's Law is the fundamental governing law of parallel processing.

Misinterpretations have led to debates and misconceptions.

Proper understanding of terms clarifies the law's universal relevance.

Abstract

The paper explains why Amdahl's Law shall be interpreted specifically for distributed parallel systems and why it generated so many debates, discussions, and abuses. We set up a general model and list many of the terms affecting parallel processing. We scrutinize the validity of neglecting certain terms in different approximations, with special emphasis on the famous scaling laws of parallel processing. We clarify that when using the right interpretation of terms, Amdahl's Law is the governing law of all kinds of parallel processing. Amdahl's Law describes among others the history of supercomputing, the inherent performance limitation of the different kinds of parallel processing and it is the basic Law of the 'modern computing' paradigm, that the computing systems working under extreme computing conditions are desperately needed.