Learning Interpretable Differentiable Logic Networks

TL;DR

This paper presents a new method for learning interpretable differentiable logic networks that combine the accuracy of neural networks with the transparency of logic, suitable for edge devices.

Contribution

The paper introduces differentiable logic networks trained via gradient methods, offering high accuracy and interpretability, with significantly reduced inference complexity.

Findings

Achieved comparable or better accuracy than traditional neural networks on twenty classification tasks.

Networks are highly interpretable due to their layered logic structure.

Inference requires up to a thousand times fewer logic gate operations, enabling edge deployment.

Abstract

The ubiquity of neural networks (NNs) in real-world applications, from healthcare to natural language processing, underscores their immense utility in capturing complex relationships within high-dimensional data. However, NNs come with notable disadvantages, such as their "black-box" nature, which hampers interpretability, as well as their tendency to overfit the training data. We introduce a novel method for learning interpretable differentiable logic networks (DLNs) that are architectures that employ multiple layers of binary logic operators. We train these networks by softening and differentiating their discrete components, e.g., through binarization of inputs, binary logic operations, and connections between neurons. This approach enables the use of gradient-based learning methods. Experimental results on twenty classification tasks indicate that differentiable logic networks can achieve accuracies comparable to or exceeding that of traditional NNs. Equally importantly, these networks offer the advantage of interpretability. Moreover, their relatively simple structure results in the number of logic gate-level operations during inference being up to a thousand times smaller than NNs, making them suitable for deployment on edge devices.