Initialization of a Polyharmonic Cascade, Launch and Testing

TL;DR

This paper introduces a universal initialization method for polyharmonic cascades, enabling stable, scalable training of deep neural networks up to 500 layers with simplified computations, demonstrated on multiple datasets.

Contribution

Proposes a novel initialization procedure for polyharmonic cascades based on symmetric hyperoctahedral constellations, enhancing stability and scalability in deep learning architectures.

Findings

Stable training of up to 500 layers without skip connections

High accuracy on MNIST, HIGGS, and Epsilon datasets

Efficient GPU implementation with reduced linear algebra operations

Abstract

This paper concludes a series of studies on the polyharmonic cascade, a deep machine learning architecture theoretically derived from indifference principles and the theory of random functions. A universal initialization procedure is proposed, based on symmetric constellations in the form of hyperoctahedra with a central point. This initialization not only ensures stable training of cascades with tens and hundreds of layers (up to 500 layers without skip connections), but also radically simplifies the computations. Scalability and robustness are demonstrated on MNIST (98.3% without convolutions or augmentations), HIGGS (AUC approximately 0.885 on 11M examples), and Epsilon (AUC approximately 0.963 with 2000 features). All linear algebra is reduced to 2D operations and is efficiently executed on GPUs. A public repository and an archived snapshot are provided for full reproducibility.