Constructing Basis Path Set by Eliminating Path Dependency

TL;DR

This paper introduces a graph-theoretic approach and an algorithm to identify basis path sets in neural networks, addressing path dependency issues and enabling broader application of G-SGD for improved network generalization.

Contribution

It formulates the basis path set searching problem, proposes the DEAH algorithm, and provides theoretical analysis to facilitate the use of G-SGD in practical neural networks.

Findings

DEAH algorithm effectively finds basis path sets.

Theoretical proofs confirm polynomial time complexity.

Methodology supports G-SGD application in complex networks.

Abstract

The way the basis path set works in neural network remains mysterious, and the generalization of newly appeared G-SGD algorithm to more practical network is hindered. The Basis Path Set Searching problem is formulated from the perspective of graph theory, to find the basis path set in a regular complicated neural network. Our paper aims to discover the underlying cause of path dependency between two independent substructures. Algorithm DEAH is designed to solve the Basis Path Set Searching problem by eliminating such path dependency. The path subdivision chain is proposed to effectively eliminate the path dependency inside the chain and between chains. The theoretical proofs and analysis of polynomial time complexity are presented. The paper therefore provides one methodology to find the basis path set in a more general neural network, which offers theoretical and algorithmic support for the application of G-SGD algorithm in more practical scenarios.