WarpCore: A Library for fast Hash Tables on GPUs
Daniel J\"unger (1), Robin Kobus (1), Andr\'e M\"uller (1), Christian, Hundt (2), Kai Xu (3), Weiguo Liu (3), Bertil Schmidt (1) ((1) Johannes, Gutenberg University, (2) NVIDIA AI Technology Center, (3) Shandong, University)

TL;DR
WarpCore is a GPU-optimized hash table library that significantly outperforms existing solutions in speed and scalability, enabling high-throughput data processing and bioinformatics applications.
Contribution
The paper introduces WarpCore, a novel GPU hash table library that achieves unprecedented performance and scalability for large-scale data processing tasks.
Findings
Achieves up to 1.6 billion inserts/sec and 4.3 billion retrievals/sec on a single GPU.
Outperforms state-of-the-art solutions like cuDPP, SlabHash, and cuDF.
Enables over two orders-of-magnitude speedup in bioinformatics applications.
Abstract
Hash tables are ubiquitous. Properties such as an amortized constant time complexity for insertion and querying as well as a compact memory layout make them versatile associative data structures with manifold applications. The rapidly growing amount of data emerging in many fields motivated the need for accelerated hash tables designed for modern parallel architectures. In this work, we exploit the fast memory interface of modern GPUs together with a parallel hashing scheme tailored to improve global memory access patterns, to design WarpCore -- a versatile library of hash table data structures. Unique device-sided operations allow for building high performance data processing pipelines entirely on the GPU. Our implementation achieves up to 1.6 billion inserts and up to 4.3 billion retrievals per second on a single GV100 GPU thereby outperforming the state-of-the-art solutions cuDPP,…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Code & Models
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
