Unbiased Weight Maximization

TL;DR

This paper introduces Unbiased Weight Maximization, a biologically plausible learning rule for neural networks that improves learning speed and scalability by replacing global reward signals with local weight norms, supported by theoretical analysis.

Contribution

It proposes Unbiased Weight Maximization, a novel unbiased learning rule that enhances learning efficiency and scalability for networks of Bernoulli-logistic units.

Findings

Unbiased Weight Maximization increases learning speed.

It scales well with network size.

It outperforms previous methods in asymptotic performance.

Abstract

A biologically plausible method for training an Artificial Neural Network (ANN) involves treating each unit as a stochastic Reinforcement Learning (RL) agent, thereby considering the network as a team of agents. Consequently, all units can learn via REINFORCE, a local learning rule modulated by a global reward signal, which aligns more closely with biologically observed forms of synaptic plasticity. Nevertheless, this learning method is often slow and scales poorly with network size due to inefficient structural credit assignment, since a single reward signal is broadcast to all units without considering individual contributions. Weight Maximization, a proposed solution, replaces a unit's reward signal with the norm of its outgoing weight, thereby allowing each hidden unit to maximize the norm of the outgoing weight instead of the global reward signal. In this research report, we analyze the theoretical properties of Weight Maximization and propose a variant, Unbiased Weight Maximization. This new approach provides an unbiased learning rule that increases learning speed and improves asymptotic performance. Notably, to our knowledge, this is the first learning rule for a network of Bernoulli-logistic units that is unbiased and scales well with the number of network's units in terms of learning speed.