Parallel Statistical Computing with R: An Illustration on Two Architectures

TL;DR

This paper demonstrates two parallel computing approaches in R for statistical tasks, illustrating their implementation and performance on multicore and cluster architectures using a random forest example.

Contribution

It introduces two parallel computing methods in R aligned with hardware realities, enhancing understanding and performance in multicore and multinode environments.

Findings

Parallel approaches effectively utilize multicore and cluster hardware.

Performance differences depend on hardware architecture.

Simple example illustrates practical implementation and benefits.

Abstract

To harness the full benefit of new computing platforms, it is necessary to develop software with parallel computing capabilities. This is no less true for statisticians than for astrophysicists. The R programming language, which is perhaps the most popular software environment for statisticians today, has many packages available for parallel computing. Their diversity in approach can be difficult to navigate. Some have attempted to alleviate this problem by designing common interfaces. However, these approaches offer limited flexibility to the user; additionally, they often serve as poor abstractions to the reality of modern hardware, leading to poor performance. We give a short introduction to two basic parallel computing approaches that closely align with hardware reality, allow the user to understand its performance, and provide sufficient capability to fully utilize multicore and multinode environments. We illustrate both approaches by working through a simple example fitting a random forest model. Beginning with a serial algorithm, we derive two parallel versions. Our objective is to illustrate the use of multiple cores on a single processor and the use of multiple processors in a cluster computer. We discuss the differences between the two versions and how the underlying hardware is used in each case.