Dimension reduction in heterogeneous neural networks: generalized Polynomial Chaos (gPC) and ANalysis-Of-VAriance (ANOVA)

TL;DR

This paper introduces two novel coarse-graining methods for large heterogeneous neural networks, leveraging uncertainty quantification tools like generalized Polynomial Chaos and ANOVA to improve simulation and analysis efficiency.

Contribution

The paper presents new approaches inspired by uncertainty quantification techniques for reducing complexity in heterogeneous neural networks.

Findings

Effective in accelerating large-scale network simulations

Able to handle structural and intrinsic heterogeneities

Facilitates coarse-grained fixed point and stability analysis

Abstract

We propose, and illustrate via a neural network example, two different approaches to coarse-graining large heterogeneous networks. Both approaches are inspired from, and use tools developed in, methods for uncertainty quantification in systems with multiple uncertain parameters - in our case, the parameters are heterogeneously distributed on the network nodes. The approach shows promise in accelerating large scale network simulations as well as coarse-grained fixed point, periodic solution and stability analysis. We also demonstrate that the approach can successfully deal with structural as well as intrinsic heterogeneities.