# Active online learning in the binary perceptron problem

## TL;DR

This paper explores active online learning in the binary perceptron, demonstrating theoretical limits and proposing Bayesian and deductive strategies for efficient error-free inference with fewer samples.

## Contribution

It introduces Bayesian and deductive pattern designing protocols that improve sample efficiency for perfect inference in the binary perceptron.

## Key findings

- Perfect inference theoretically possible with N samples but computationally infeasible.
- Bayesian protocols require approximately 2.3 N and 1.9 N samples for error-free inference.
- Deductive reasoning achieves perfect inference with N + log2 N samples.

## Abstract

The binary perceptron is the simplest artificial neural network formed by $N$ input units and one output unit, with the neural states and the synaptic weights all restricted to $\pm 1$ values. The task in the teacher--student scenario is to infer the hidden weight vector by training on a set of labeled patterns. Previous efforts on the passive learning mode have shown that learning from independent random patterns is quite inefficient. Here we consider the active online learning mode in which the student designs every new Ising training pattern. We demonstrate that it is mathematically possible to achieve perfect (error-free) inference using only $N$ designed training patterns, but this is computationally unfeasible for large systems. We then investigate two Bayesian statistical designing protocols, which require $2.3 N$ and $1.9 N$ training patterns, respectively, to achieve error-free inference. If the training patterns are instead designed through deductive reasoning, perfect inference is achieved using $N\!+\!\log_{2}\!N$ samples. The performance gap between Bayesian and deductive designing strategies may be shortened in future work by taking into account the possibility of ergodicity breaking in the version space of the binary perceptron.