Adaptive Learning with Binary Neurons

TL;DR

This paper introduces NetLines, an efficient incremental learning algorithm that constructs compact neural networks with binary units, demonstrating competitive performance and convergence guarantees for classification tasks with binary and real-valued inputs.

Contribution

The paper presents NetLines, a novel incremental learning algorithm for binary neural networks with proven convergence and applicability to multi-class classification.

Findings

NetLines produces small, efficient neural networks.

The algorithm converges with a finite number of hidden units.

Early stopping reduces overfitting without affecting generalization.

Abstract

A efficient incremental learning algorithm for classification tasks, called NetLines, well adapted for both binary and real-valued input patterns is presented. It generates small compact feedforward neural networks with one hidden layer of binary units and binary output units. A convergence theorem ensures that solutions with a finite number of hidden units exist for both binary and real-valued input patterns. An implementation for problems with more than two classes, valid for any binary classifier, is proposed. The generalization error and the size of the resulting networks are compared to the best published results on well-known classification benchmarks. Early stopping is shown to decrease overfitting, without improving the generalization performance.