KLay: Accelerating Arithmetic Circuits for Neurosymbolic AI
Jaron Maene, Vincent Derkinderen, Pedro Zuidberg Dos Martires
TL;DR
KLay introduces a novel data structure and algorithms to efficiently parallelize arithmetic circuits on GPUs, significantly accelerating neurosymbolic AI computations and enabling larger-scale applications.
Contribution
The paper presents KLay, a new data structure and algorithms that enable efficient GPU parallelization of arithmetic circuits for neurosymbolic AI.
Findings
Achieves multiple orders of magnitude speedup over existing methods
Enables scaling neurosymbolic AI to larger, real-world problems
Provides algorithms for translating and evaluating circuits efficiently
Abstract
A popular approach to neurosymbolic AI involves mapping logic formulas to arithmetic circuits (computation graphs consisting of sums and products) and passing the outputs of a neural network through these circuits. This approach enforces symbolic constraints onto a neural network in a principled and end-to-end differentiable way. Unfortunately, arithmetic circuits are challenging to run on modern AI accelerators as they exhibit a high degree of irregular sparsity. To address this limitation, we introduce knowledge layers (KLay), a new data structure to represent arithmetic circuits that can be efficiently parallelized on GPUs. Moreover, we contribute two algorithms used in the translation of traditional circuit representations to KLay and a further algorithm that exploits parallelization opportunities during circuit evaluations. We empirically show that KLay achieves speedups of…
Peer Reviews
Decision·ICLR 2025 Poster
First, the paper tackles an important challenge in neuro-symbolic AI, where scalability is still one of the major roadblocks for their general application. The achieved speed-ups of the proposed KLay on both CPU and GPU seem very promising. Finally, the paper is well-written, and the methods are explained intuitively thanks to the use of appropriate figures and examples. This makes the paper accessible despite the methodology being quite involved.
1) Although the paper indicated in 3rd paragraph that it focused on a "particular flavor of neurosymbolic AI", i.e., a neural network feeding into probabilistic inference based on arithmetic circuits, it would be beneficial to reflect it explicitly in other parts, e.g., in the abstract, or a more specific title (Accelerating Arithmetic Circuits in Neurosymbolic AI), to be able to attract the right audience. 2) The actual "interface" and details between the neural network and the used arithmetic
1. The utilization of modern AI accelerators is crucial for advancing the field of neurosymbolic AI, enabling the development of large-scale models and tackling more complex problems that have been previously out of reach. 2. The ability to enable parallel computation on GPUs results in speedups of multiple orders of magnitude over current state-of-the-art approaches.
1. The background introduction to neurosymbolic AI is somewhat lacking, providing insufficient context. 2. The experimental comparisons with state-of-the-art methods appear limited in scope compared to those discussed in related works.
The problem the authors formulated is clear.
The motivation needs further justification.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsNeural Networks and Applications