Quantile Mechanics II: Changes of Variables in Monte Carlo methods and GPU-Optimized Normal Quantiles

TL;DR

This paper introduces a novel change-of-variables method for Monte Carlo sampling that enhances GPU efficiency and precision, especially for normal quantiles, by avoiding branching and warp divergence.

Contribution

It develops a new analytical approach for distribution transformations, improving GPU-based Monte Carlo sampling accuracy and performance over existing methods.

Findings

Single-approximation normal quantiles achieve high accuracy across wide ranges.

GPU implementations outperform traditional methods in speed and precision.

The approach is effective in both single and double precision modes.

Abstract

This article presents differential equations and solution methods for the functions of the form $Q(x) = F^{-1}(G(x))$, where $F$ and $G$ are cumulative distribution functions. Such functions allow the direct recycling of Monte Carlo samples from one distribution into samples from another. The method may be developed analytically for certain special cases, and illuminate the idea that it is a more precise form of the traditional Cornish-Fisher expansion. In this manner the model risk of distributional risk may be assessed free of the Monte Carlo noise associated with resampling. Examples are given of equations for converting normal samples to Student t, and converting exponential to hyperbolic, variance gamma and normal. In the case of the normal distribution, the change of variables employed allows the sampling to take place to good accuracy based on a single rational approximation over a very wide range of the sample space. The avoidance of any branching statement is of use in optimal GPU computations as it avoids the effect of {\it warp divergence}, and we give examples of branch-free normal quantiles that offer performance improvements in a GPU environment, while retaining the best precision characteristics of well-known methods. We also offer models based on a low-probability of warp divergence. Comparisons of new and old forms are made on the Nvidia Quadro 4000, GTX 285 and 480, and Tesla C2050 GPUs. We argue that in single-precision mode, the change-of-variables approach offers performance competitive with the fastest existing scheme while substantially improving precision, and that in double-precision mode, this approach offers the most GPU-optimal Gaussian quantile yet, and without compromise on precision for Monte Carlo applications, working twice as fast as the CUDA 4 library function with increased precision.