Two Instances of Interpretable Neural Network for Universal Approximations

TL;DR

This paper introduces two interpretable neural network models, TNN and SQANN, that achieve universal approximation, resist catastrophic forgetting, and can identify out-of-distribution samples through interpretable activation patterns.

Contribution

The paper presents two novel bottom-up interpretable neural network constructions, TNN and SQANN, with proven high accuracy and capabilities for out-of-distribution detection.

Findings

Achieved universal approximation with TNN and SQANN

Resistant to catastrophic forgetting

Able to identify out-of-distribution samples

Abstract

This paper proposes two bottom-up interpretable neural network (NN) constructions for universal approximation, namely Triangularly-constructed NN (TNN) and Semi-Quantized Activation NN (SQANN). Further notable properties are (1) resistance to catastrophic forgetting (2) existence of proof for arbitrarily high accuracies (3) the ability to identify samples that are out-of-distribution through interpretable activation "fingerprints".