Large Scale Parallel Computations in R through Elemental

TL;DR

This paper introduces RElem, an R package that integrates the Elemental distributed linear algebra library, enabling scalable high-performance parallel computations in R for large-scale statistical analysis.

Contribution

It presents a novel R package that seamlessly combines R with high-performance distributed linear algebra, facilitating large-scale statistical computations.

Findings

RElem enables R to perform distributed dense linear algebra operations.

The package allows easy porting of existing R programs to distributed environments.

RElem achieves high performance comparable to native Elemental computations without overhead.

Abstract

Even though in recent years the scale of statistical analysis problems has increased tremendously, many statistical software tools are still limited to single-node computations. However, statistical analyses are largely based on dense linear algebra operations, which have been deeply studied, optimized and parallelized in the high-performance-computing community. To make high-performance distributed computations available for statistical analysis, and thus enable large scale statistical computations, we introduce RElem, an open source package that integrates the distributed dense linear algebra library Elemental into R. While on the one hand, RElem provides direct wrappers of Elemental's routines, on the other hand, it overloads various operators and functions to provide an entirely native R experience for distributed computations. We showcase how simple it is to port existing R programs to Relem and demonstrate that Relem indeed allows to scale beyond the single-node limitation of R with the full performance of Elemental without any overhead.