To prune or not to prune : A chaos-causality approach to principled pruning of dense neural networks

TL;DR

This paper introduces a novel chaos-causality based approach to prune dense neural networks by identifying causal weights responsible for misclassification, aiming to reduce size without performance loss.

Contribution

It formulates pruning as an optimization problem using chaos theory and causality, providing a new method to prune networks while preserving accuracy and explainability.

Findings

Pruned networks maintain original performance.

The method identifies causal weights responsible for errors.

Pruning retains feature explainability.

Abstract

Reducing the size of a neural network (pruning) by removing weights without impacting its performance is an important problem for resource-constrained devices. In the past, pruning was typically accomplished by ranking or penalizing weights based on criteria like magnitude and removing low-ranked weights before retraining the remaining ones. Pruning strategies may also involve removing neurons from the network in order to achieve the desired reduction in network size. We formulate pruning as an optimization problem with the objective of minimizing misclassifications by selecting specific weights. To accomplish this, we have introduced the concept of chaos in learning (Lyapunov exponents) via weight updates and exploiting causality to identify the causal weights responsible for misclassification. Such a pruned network maintains the original performance and retains feature explainability.