On the Perceptron's Compression

TL;DR

This paper explores the perceptron's compression phenomena, proposing algorithm modifications for improved margin guarantees and demonstrating their potential benefits in neural network training, supported by experimental data.

Contribution

It introduces modifications to the perceptron algorithm that enhance margin guarantees and applies these insights to neural network training contexts.

Findings

Modified perceptron algorithms improve margin guarantees

Experimental data supports benefits in neural network training

Insights applicable to various learning contexts

Abstract

We study and provide exposition to several phenomena that are related to the perceptron's compression. One theme concerns modifications of the perceptron algorithm that yield better guarantees on the margin of the hyperplane it outputs. These modifications can be useful in training neural networks as well, and we demonstrate them with some experimental data. In a second theme, we deduce conclusions from the perceptron's compression in various contexts.