Polarity is all you need to learn and transfer faster

TL;DR

This paper explores how the initial setting of weight polarities in neural networks influences their learning speed and efficiency, showing that proper polarity initialization can significantly reduce training time and data requirements.

Contribution

The study introduces the concept that weight polarity initialization is a key factor in neural network learning efficiency, supported by simulations and classification experiments.

Findings

Proper polarity initialization accelerates learning.

Incorrect polarity settings can hinder network performance.

Weight polarities influence data and computational efficiency.

Abstract

Natural intelligences (NIs) thrive in a dynamic world - they learn quickly, sometimes with only a few samples. In contrast, artificial intelligences (AIs) typically learn with a prohibitive number of training samples and computational power. What design principle difference between NI and AI could contribute to such a discrepancy? Here, we investigate the role of weight polarity: development processes initialize NIs with advantageous polarity configurations; as NIs grow and learn, synapse magnitudes update, yet polarities are largely kept unchanged. We demonstrate with simulation and image classification tasks that if weight polarities are adequately set a priori, then networks learn with less time and data. We also explicitly illustrate situations in which a priori setting the weight polarities is disadvantageous for networks. Our work illustrates the value of weight polarities from the perspective of statistical and computational efficiency during learning.