Analysis of function approximation and stability of general DNNs in directed acyclic graphs using un-rectifying analysis

TL;DR

This paper introduces an un-rectifying analysis method for general DNNs modeled as directed acyclic graphs, providing new insights into their function approximation and stability properties.

Contribution

It develops a bottom-up axiomatic approach to analyze DNNs via DAGs, enabling derivation of properties for a broad class of neural networks.

Findings

Properties of general DNNs can be derived using the proposed analytical graph approach.

The method advances understanding of network functions and stability in DNNs.

Analytical tools for graphs can be leveraged for further theoretical insights.

Abstract

A general lack of understanding pertaining to deep feedforward neural networks (DNNs) can be attributed partly to a lack of tools with which to analyze the composition of non-linear functions, and partly to a lack of mathematical models applicable to the diversity of DNN architectures. In this paper, we made a number of basic assumptions pertaining to activation functions, non-linear transformations, and DNN architectures in order to use the un-rectifying method to analyze DNNs via directed acyclic graphs (DAGs). DNNs that satisfy these assumptions are referred to as general DNNs. Our construction of an analytic graph was based on an axiomatic method in which DAGs are built from the bottom-up through the application of atomic operations to basic elements in accordance with regulatory rules. This approach allows us to derive the properties of general DNNs via mathematical induction. We show that using the proposed approach, some properties hold true for general DNNs can be derived. This analysis advances our understanding of network functions and could promote further theoretical insights if the host of analytical tools for graphs can be leveraged.