QuPARA: Query-Driven Large-Scale Portfolio Aggregate Risk Analysis on MapReduce

TL;DR

This paper presents QuPARA, a Hadoop-based framework enabling flexible, query-driven portfolio risk analysis on large-scale data, allowing detailed ad hoc queries beyond traditional aggregated risk metrics.

Contribution

The paper introduces QuPARA, a novel system that leverages MapReduce for efficient, flexible, large-scale portfolio risk analysis with support for unanticipated queries.

Findings

Can analyze a million trials in just over 20 minutes on a 16-node cluster.

Supports detailed, ad hoc risk queries beyond standard metrics.

Demonstrates feasibility of large-scale, flexible risk analysis using Hadoop.

Abstract

Stochastic simulation techniques are used for portfolio risk analysis. Risk portfolios may consist of thousands of reinsurance contracts covering millions of insured locations. To quantify risk each portfolio must be evaluated in up to a million simulation trials, each capturing a different possible sequence of catastrophic events over the course of a contractual year. In this paper, we explore the design of a flexible framework for portfolio risk analysis that facilitates answering a rich variety of catastrophic risk queries. Rather than aggregating simulation data in order to produce a small set of high-level risk metrics efficiently (as is often done in production risk management systems), the focus here is on allowing the user to pose queries on unaggregated or partially aggregated data. The goal is to provide a flexible framework that can be used by analysts to answer a wide variety of unanticipated but natural ad hoc queries. Such detailed queries can help actuaries or underwriters to better understand the multiple dimensions (e.g., spatial correlation, seasonality, peril features, construction features, and financial terms) that can impact portfolio risk. We implemented a prototype system, called QuPARA (Query-Driven Large-Scale Portfolio Aggregate Risk Analysis), using Hadoop, which is Apache's implementation of the MapReduce paradigm. This allows the user to take advantage of large parallel compute servers in order to answer ad hoc risk analysis queries efficiently even on very large data sets typically encountered in practice. We describe the design and implementation of QuPARA and present experimental results that demonstrate its feasibility. A full portfolio risk analysis run consisting of a 1,000,000 trial simulation, with 1,000 events per trial, and 3,200 risk transfer contracts can be completed on a 16-node Hadoop cluster in just over 20 minutes.