Functional Network: A Novel Framework for Interpretability of Deep Neural Networks

TL;DR

This paper introduces a novel interpretability framework for deep neural networks called the functional network, utilizing graph analysis to reveal how regularization methods affect model behavior and robustness.

Contribution

It proposes a new functional network framework inspired by brain networks, applying graph theory and topological data analysis to interpret deep neural networks.

Findings

Batch normalization increases global efficiency but reduces adversarial robustness.

Dropout enhances generalization and fault tolerance.

Functional topological differences cluster models by regularization type.

Abstract

The layered structure of deep neural networks hinders the use of numerous analysis tools and thus the development of its interpretability. Inspired by the success of functional brain networks, we propose a novel framework for interpretability of deep neural networks, that is, the functional network. We construct the functional network of fully connected networks and explore its small-worldness. In our experiments, the mechanisms of regularization methods, namely, batch normalization and dropout, are revealed using graph theoretical analysis and topological data analysis. Our empirical analysis shows the following: (1) Batch normalization enhances model performance by increasing the global e ciency and the number of loops but reduces adversarial robustness by lowering the fault tolerance. (2) Dropout improves generalization and robustness of models by improving the functional specialization and fault tolerance. (3) The models with dierent regularizations can be clustered correctly according to their functional topological dierences, re ecting the great potential of the functional network and topological data analysis in interpretability.