GPU Implementations for Midsize Integer Addition and Multiplication

TL;DR

This paper demonstrates simple, efficient GPU implementations for midsize integer addition and multiplication, outperforming existing libraries for large integers and exploring the use of FFT multiplication within CUDA constraints.

Contribution

It introduces straightforward GPU algorithms for integer addition and multiplication, including FFT-based methods, with performance advantages over prior complex tiling strategies.

Findings

CUDA implementation outperforms CGBN for integers >32K bits

FFT multiplication surpasses classical methods at large sizes within a CUDA block

Simple, locality-aware strategies enable efficient midsize integer arithmetic on GPUs

Abstract

This paper explores practical aspects of using a high-level functional language for GPU-based arithmetic on ``midsize'' integers. By this we mean integers of up to about a quarter million bits, which is sufficient for most practical purposes. The goal is to understand whether it is possible to support efficient nested-parallel programs with a small, flexible code base. We report on GPU implementations for addition and multiplication of integers that fit in one CUDA block, thus leveraging temporal reuse from scratchpad memories. Our key contribution resides in the simplicity of the proposed solutions: We recognize that addition is a straightforward application of scan, which is known to allow efficient GPU implementation. For quadratic multiplication we employ a simple work-partitioning strategy that offers good temporal locality. For FFT multiplication, we efficiently map the computation in the domain of integral fields by finding ``good'' primes that enable almost-full utilization of machine words. In comparison, related work uses complex tiling strategies -- which feel too big a hammer for the job -- or uses the computational domain of reals, which may degrade the magnitude of the base in which the computation is carried. We evaluate the performance in comparison to the state-of-the-art CGBN library, authored by NvidiaLab, and report that our CUDA prototype outperforms CGBN for integer sizes higher than 32K bits, while offering comparable performance for smaller sizes. Moreover, we are, to our knowledge, the first to report that FFT multiplication outperforms the classical one on the larger sizes that still fit in a CUDA block. Finally, we examine Futhark's strengths and weaknesses for efficiently supporting such computations and find out that a compiler pass aimed at efficient sequentialization of excess parallelism would significantly improve performance.