Are Clouds Ready to Accelerate Ad hoc Financial Simulations?

TL;DR

This paper investigates whether cloud computing platforms can effectively accelerate ad hoc financial risk simulations, demonstrating significant speedups with hardware accelerators but also highlighting cost inefficiencies and private cloud advantages.

Contribution

The study develops and deploys an 'Aggregate Risk Analysis' simulation on the cloud, exploring parallel techniques and hardware accelerators to improve performance.

Findings

Achieved up to 24x acceleration with multiple hardware accelerators.

Obtained approximately 60x speedup over baseline implementations.

Private VMs outperform public VMs with similar hardware.

Abstract

Applications employed in the financial services industry to capture and estimate a variety of risk metrics are underpinned by stochastic simulations which are data, memory and computationally intensive. Many of these simulations are routinely performed on production-based computing systems. Ad hoc simulations in addition to routine simulations are required to obtain up-to-date views of risk metrics. Such simulations are currently not performed as they cannot be accommodated on production clusters, which are typically over committed resources. Scalable, on-demand and pay-as-you go Virtual Machines (VMs) offered by the cloud are a potential platform to satisfy the data, memory and computational constraints of the simulation. However, "Are clouds ready to accelerate ad hoc financial simulations?" The research reported in this paper aims to experimentally verify this question by developing and deploying an important financial simulation, referred to as 'Aggregate Risk Analysis' on the cloud. Parallel techniques to improve efficiency and performance of the simulations are explored. Challenges such as accommodating large input data on limited memory VMs and rapidly processing data for real-time use are surmounted. The key result of this investigation is that Aggregate Risk Analysis can be accommodated on cloud VMs. Acceleration of up to 24x using multiple hardware accelerators over the implementation on a single accelerator, 6x over a multiple core implementation and approximately 60x over a baseline implementation was achieved on the cloud. However, computational time is wasted for every dollar spent on the cloud due to poor acceleration over multiple virtual cores. Interestingly, private VMs can offer better performance than public VMs on comparable underlying hardware.