Fuzzy Logic Interpretation of Quadratic Networks

TL;DR

This paper explores how quadratic neural networks can be interpreted as deep fuzzy logic systems, providing a framework for understanding their operations through fuzzy logic principles and analyzing their information properties.

Contribution

It generalizes fuzzy logic operations in quadratic neurons and offers a statistical analysis of their information-theoretic characteristics.

Findings

Quadratic neurons can implement various fuzzy logic operations.

Deep quadratic networks can be interpreted as fuzzy logic systems.

Statistical analysis reveals information processing capabilities.

Abstract

Over past several years, deep learning has achieved huge successes in various applications. However, such a data-driven approach is often criticized for lack of interpretability. Recently, we proposed artificial quadratic neural networks consisting of second-order neurons in potentially many layers. In each second-order neuron, a quadratic function is used in the place of the inner product in a traditional neuron, and then undergoes a nonlinear activation. With a single second-order neuron, any fuzzy logic operation, such as XOR, can be implemented. In this sense, any deep network constructed with quadratic neurons can be interpreted as a deep fuzzy logic system. Since traditional neural networks and second-order counterparts can represent each other and fuzzy logic operations are naturally implemented in second-order neural networks, it is plausible to explain how a deep neural network works with a second-order network as the system model. In this paper, we generalize and categorize fuzzy logic operations implementable with individual second-order neurons, and then perform statistical/information theoretic analyses of exemplary quadratic neural networks.