Interpretable neural networks based on continuous-valued logic and multicriteria decision operators

TL;DR

This paper introduces a hybrid neural network architecture based on continuous-valued logic and multicriteria decision operators, enhancing interpretability and reducing parameters by integrating logical structures into the network design.

Contribution

It presents a novel framework using nilpotent logical systems to combine continuous logic with neural networks, improving interpretability and parameter efficiency.

Findings

Reduces number of learnable parameters significantly.

Provides a theoretical basis for activation functions including ReLU.

Demonstrates the architecture with toy examples from tensorflow playground.

Abstract

Combining neural networks with continuous logic and multicriteria decision making tools can reduce the black box nature of neural models. In this study, we show that nilpotent logical systems offer an appropriate mathematical framework for a hybridization of continuous nilpotent logic and neural models, helping to improve the interpretability and safety of machine learning. In our concept, perceptrons model soft inequalities; namely membership functions and continuous logical operators. We design the network architecture before training, using continuous logical operators and multicriteria decision tools with given weights working in the hidden layers. Designing the structure appropriately leads to a drastic reduction in the number of parameters to be learned. The theoretical basis offers a straightforward choice of activation functions (the cutting function or its differentiable approximation, the squashing function), and also suggests an explanation to the great success of the rectified linear unit (ReLU). In this study, we focus on the architecture of a hybrid model and introduce the building blocks for future application in deep neural networks. The concept is illustrated with some toy examples taken from an extended version of the tensorflow playground.