Julia Cloud Matrix Machine: Dynamic Matrix Language Acceleration on Multicore Clusters in the Cloud

TL;DR

This paper introduces a cloud-optimized extension of Julia that automatically parallelizes large matrix computations across multicore clusters, achieving significant speedups and efficient resource utilization.

Contribution

It presents a novel cloud-aware extension of Julia with automatic parallelization, lazy evaluation, and dynamic scheduling for matrix computations in cloud environments.

Findings

Achieved up to 5.1x speedup on 14-node AWS cluster

Average speedup of 74.39% of the theoretical maximum

Demonstrated effective scaling and resource utilization in cloud settings

Abstract

In emerging scientific computing environments, matrix computations of increasing size and complexity are increasingly becoming prevalent. However, contemporary matrix language implementations are insufficient in their support for efficient utilization of cloud computing resources, particularly on the user side. We thus developed an extension of the Julia high-performance computation language such that matrix computations are automatically parallelized in the cloud, where users are separated from directly interacting with complex explicitly-parallel computations. We implement lazy evaluation semantics combined with directed graphs to optimize matrix operations on the fly while dynamic simulation finds the optimal tile size and schedule for a given cluster of cloud nodes. A time model prediction of the cluster's performance capacity is constructed to enable simulations. Automatic configuration of communication and worker processes on the cloud networks allow for the framework to automatically scale up for clusters of heterogeneous nodes. Our framework's experimental evaluation comprises eleven benchmarks on an fourteen node (564 CPUs) cluster in the AWS public cloud, revealing speedups of up to a factor of 5.1, with an average 74.39% of the upper bound for speedups.