# Controlling Covariate Shift using Balanced Normalization of Weights

## TL;DR

This paper presents a novel weight normalization method that balances positive and negative weights to improve convergence speed, validated on standard image classification benchmarks.

## Contribution

The paper introduces a new normalization technique that normalizes layer weights to achieve fast convergence similar to batch normalization.

## Key findings

- Effective on CIFAR-10/100, SVHN, and ImageNet
- Balances positive and negative weight contributions
- Achieves rapid convergence comparable to batch normalization

## Abstract

We introduce a new normalization technique that exhibits the fast convergence properties of batch normalization using a transformation of layer weights instead of layer outputs. The proposed technique keeps the contribution of positive and negative weights to the layer output balanced. We validate our method on a set of standard benchmarks including CIFAR-10/100, SVHN and ILSVRC 2012 ImageNet.