Accurate Computation of the Logarithm of Modified Bessel Functions on GPUs

TL;DR

This paper introduces two novel algorithms for accurately computing the logarithm of modified Bessel functions on GPUs, significantly improving precision and speed over existing methods, enabling advanced applications like high-dimensional neural network feature analysis.

Contribution

The paper presents robust, fast algorithms for computing the log of modified Bessel functions with machine precision accuracy, outperforming existing libraries and enabling new high-dimensional data applications.

Findings

Algorithms achieve up to 6150x speedup on GPU

Relative errors are on the order of machine precision

Successfully fit von Mises-Fisher distributions to high-dimensional data

Abstract

Bessel functions are critical in scientific computing for applications such as machine learning, protein structure modeling, and robotics. However, currently, available routines lack precision or fail for certain input ranges, such as when the order $v$ is large, and GPU-specific implementations are limited. We address the precision limitations of current numerical implementations while dramatically improving the runtime. We propose two novel algorithms for computing the logarithm of modified Bessel functions of the first and second kinds by computing intermediate values on a logarithmic scale. Our algorithms are robust and never have issues with underflows or overflows while having relative errors on the order of machine precision, even for inputs where existing libraries fail. In C++/CUDA, our algorithms have median and maximum speedups of 45x and 6150x for GPU and 17x and 3403x for CPU, respectively, over the ranges of inputs and third-party libraries tested. Compared to SciPy, the algorithms have median and maximum speedups of 77x and 300x for GPU and 35x and 98x for CPU, respectively, over the tested inputs. The ability to robustly compute a solution and the low relative errors allow us to fit von Mises-Fisher, vMF, distributions to high-dimensional neural network features. This is, e.g., relevant for uncertainty quantification in metric learning. We obtain image feature data by processing CIFAR10 training images with the convolutional layers of a pre-trained ResNet50. We successfully fit vMF distributions to 2048-, 8192-, and 32768-dimensional image feature data using our algorithms. Our approach provides fast and accurate results while existing implementations in SciPy and mpmath fail to fit successfully. Our approach is readily implementable on GPUs, and we provide a fast open-source implementation alongside this paper.