Median Binary-Connect Method and a Binary Convolutional Neural Nework for Word Recognition

TL;DR

This paper introduces a median-based projection method for training binary convolutional neural networks, improving accuracy and speed in keyword recognition tasks compared to traditional methods.

Contribution

It proposes a new l_1 norm-based projection formula using median computation, enhancing binary network training and performance.

Findings

Median BC outperforms regular BC in experiments.

Binary network achieves 92.4% accuracy, close to full-precision.

Doubles speed of spoken keyword recognition on Android.

Abstract

We propose and study a new projection formula for training binary weight convolutional neural networks. The projection formula measures the error in approximating a full precision (32 bit) vector by a 1-bit vector in the l_1 norm instead of the standard l_2 norm. The l_1 projector is in closed analytical form and involves a median computation instead of an arithmatic average in the l_2 projector. Experiments on 10 keywords classification show that the l_1 (median) BinaryConnect (BC) method outperforms the regular BC, regardless of cold or warm start. The binary network trained by median BC and a recent blending technique reaches test accuracy 92.4%, which is 1.1% lower than the full-precision network accuracy 93.5%. On Android phone app, the trained binary network doubles the speed of full-precision network in spoken keywords recognition.