Neural networks as fuzzy logic formulas

TL;DR

This paper characterizes neural networks with ReLU activations using fuzzy logic, linking their expressive power to well-established fuzzy logics and extending logical frameworks to include arbitrary real activation values.

Contribution

It provides the first fuzzy logic characterizations of rational-weight ReLU neural networks and extends logical frameworks to encompass networks with arbitrary real activations.

Findings

Fuzzy logic characterizations of ReLU neural networks are established.

Logical frameworks are extended to include arbitrary real activation values.

Connections between neural network expressiveness and fuzzy logic are demonstrated.

Abstract

Neural networks are a fundamental aspect of modern artificial intelligence, playing a key role in various important machine learning architectures including transformers and graph neural networks. Recently, logical characterisations have been used to study the expressive power of many machine learning architectures, but logical characterisations of plain neural networks have received less attention. In this paper, we provide fuzzy logic characterisations of rational-weight ReLU-activated neural networks via two well-established fuzzy logics: Rational Pavelka Logic RPL (and extensions thereof) and (fragments of) $\mathit{L \Pi} \frac{1}{2}$. The activation values of the neural networks are allowed to be arbitrary real numbers. We also provide fuzzy logic characterisations of a generalised polynomial ring over $\mathbb{Q}$ in countably many variables where the use of the ReLU-function is permitted.