ANDHRA Bandersnatch: Training Neural Networks to Predict Parallel Realities

TL;DR

This paper introduces ANDHRA, a neural network architecture inspired by the Many-Worlds Interpretation, which creates parallel branches at each layer to improve accuracy without increasing inference cost.

Contribution

The novel ANDHRA architecture employs parallel branching with Hyper Rectified Activation, enabling multiple network paths that enhance accuracy while maintaining computational efficiency.

Findings

One branch outperforms baseline accuracy on CIFAR datasets.

Parallel branches achieve significant accuracy improvements.

Additional parameters are only needed during training.

Abstract

Inspired by the Many-Worlds Interpretation (MWI), this work introduces a novel neural network architecture that splits the same input signal into parallel branches at each layer, utilizing a Hyper Rectified Activation, referred to as ANDHRA. The branched layers do not merge and form separate network paths, leading to multiple network heads for output prediction. For a network with a branching factor of 2 at three levels, the total number of heads is 2^3 = 8 . The individual heads are jointly trained by combining their respective loss values. However, the proposed architecture requires additional parameters and memory during training due to the additional branches. During inference, the experimental results on CIFAR-10/100 demonstrate that there exists one individual head that outperforms the baseline accuracy, achieving statistically significant improvement with equal parameters and computational cost.