NN2Rules: Extracting Rule List from Neural Networks

TL;DR

NN2Rules is an algorithm that converts trained neural networks into interpretable rule lists, maintaining the same accuracy and accommodating different activation functions, thus enhancing interpretability of neural models.

Contribution

It introduces a decompositional method for rule extraction from neural networks that works with both sigmoid and ReLU activations, improving interpretability.

Findings

Extracted rules match neural network predictions.

Applicable to networks with sigmoid and ReLU activations.

Maintains the same accuracy as the original neural network.

Abstract

We present an algorithm, NN2Rules, to convert a trained neural network into a rule list. Rule lists are more interpretable since they align better with the way humans make decisions. NN2Rules is a decompositional approach to rule extraction, i.e., it extracts a set of decision rules from the parameters of the trained neural network model. We show that the decision rules extracted have the same prediction as the neural network on any input presented to it, and hence the same accuracy. A key contribution of NN2Rules is that it allows hidden neuron behavior to be either soft-binary (eg. sigmoid activation) or rectified linear (ReLU) as opposed to existing decompositional approaches that were developed with the assumption of soft-binary activation.