Binarized Weight Error Networks With a Transition Regularization Term

TL;DR

This paper introduces a binarized weight network with a transition regularization term that enhances stability and capacity, reducing computation load while improving performance on various visual tasks.

Contribution

The paper presents a novel binarized weight network with a transition regularization term that improves stability and representation capacity, especially for shallow networks.

Findings

Outperforms existing binary networks on multiple datasets.

Enhances stability and capacity of shallow networks.

Reduces computational load while maintaining high accuracy.

Abstract

This paper proposes a novel binarized weight network (BT) for a resource-efficient neural structure. The proposed model estimates a binary representation of weights by taking into account the approximation error with an additional term. This model increases representation capacity and stability, particularly for shallow networks, while the computation load is theoretically reduced. In addition, a novel regularization term is introduced that is suitable for all threshold-based binary precision networks. This term penalizes the trainable parameters that are far from the thresholds at which binary transitions occur. This step promotes a swift modification for binary-precision responses at train time. The experimental results are carried out for two sets of tasks: visual classification and visual inverse problems. Benchmarks for Cifar10, SVHN, Fashion, ImageNet2012, Set5, Set14, Urban and BSD100 datasets show that our method outperforms all counterparts with binary precision.