Bisimulations for Neural Network Reduction

TL;DR

This paper introduces bisimulation concepts for neural network reduction, offering exact and approximate methods to create smaller, semantically similar networks with quantifiable deviations, balancing reduction size and accuracy.

Contribution

It proposes a formal bisimulation framework for neural network reduction, including algorithms for exact and approximate reductions with semantic deviation quantification.

Findings

Exact bisimulation yields minimal equivalent networks.

Approximate bisimulation allows controlled semantic deviations.

Trade-off between reduction size and semantic accuracy.

Abstract

We present a notion of bisimulation that induces a reduced network which is semantically equivalent to the given neural network. We provide a minimization algorithm to construct the smallest bisimulation equivalent network. Reductions that construct bisimulation equivalent neural networks are limited in the scale of reduction. We present an approximate notion of bisimulation that provides semantic closeness, rather than, semantic equivalence, and quantify semantic deviation between the neural networks that are approximately bisimilar. The latter provides a trade-off between the amount of reduction and deviations in the semantics.