SPINE: Soft Piecewise Interpretable Neural Equations

TL;DR

SPINE introduces a novel neural network architecture that uses set operations on pieces to achieve high interpretability, flexibility, and simplicity, matching the performance of traditional models on various datasets.

Contribution

The paper proposes a new interpretable neural network model based on set operations and canonical normal forms, enabling flexible, targeted, and simple piecewise function fitting.

Findings

Achieves interpretability with parameters linked to function pieces

Fits complex functions using set operations on pieces

Performs comparably to standard architectures on benchmarks

Abstract

Relu Fully Connected Networks are ubiquitous but uninterpretable because they fit piecewise linear functions emerging from multi-layered structures and complex interactions of model weights. This paper takes a novel approach to piecewise fits by using set operations on individual pieces(parts). This is done by approximating canonical normal forms and using the resultant as a model. This gives special advantages like (a)strong correspondence of parameters to pieces of the fit function(High Interpretability); (b)ability to fit any combination of continuous functions as pieces of the piecewise function(Ease of Design); (c)ability to add new non-linearities in a targeted region of the domain(Targeted Learning); (d)simplicity of an equation which avoids layering. It can also be expressed in the general max-min representation of piecewise linear functions which gives theoretical ease and credibility. This architecture is tested on simulated regression and classification tasks and benchmark datasets including UCI datasets, MNIST, FMNIST, and CIFAR 10. This performance is on par with fully connected architectures. It can find a variety of applications where fully connected layers must be replaced by interpretable layers.