Geometric Decomposition of Feed Forward Neural Networks

TL;DR

This paper introduces a geometric framework for understanding feed forward neural networks, aiming to enhance analysis, training, and application development by revealing inherent structural properties.

Contribution

It presents a novel geometric decomposition of neural networks, providing a new perspective that differs from traditional functional approaches.

Findings

Defines an inherent geometric structure in feed forward neural networks

Provides a framework for analyzing network homology and training algorithms

Lays groundwork for future geometric-based neural network research

Abstract

There have been several attempts to mathematically understand neural networks and many more from biological and computational perspectives. The field has exploded in the last decade, yet neural networks are still treated much like a black box. In this work we describe a structure that is inherent to a feed forward neural network. This will provide a framework for future work on neural networks to improve training algorithms, compute the homology of the network, and other applications. Our approach takes a more geometric point of view and is unlike other attempts to mathematically understand neural networks that rely on a functional perspective.