Topological Understanding of Neural Networks, a survey

TL;DR

This survey explores the internal topological structure of neural networks, analyzing activation functions, architectures, and empirical data to better understand their decision processes and suggest future research directions.

Contribution

It provides a comprehensive review of neural network topologies, highlighting the significance of different activation functions and proposing experimental avenues for real dataset analysis.

Findings

Insights into activation function roles

Correlation between architecture and topology

Proposals for future experimental verification

Abstract

We look at the internal structure of neural networks which is usually treated as a black box. The easiest and the most comprehensible thing to do is to look at a binary classification and try to understand the approach a neural network takes. We review the significance of different activation functions, types of network architectures associated to them, and some empirical data. We find some interesting observations and a possibility to build upon the ideas to verify the process for real datasets. We suggest some possible experiments to look forward to in three different directions.