The Attentive Perceptron
Raphael Pelossof, Zhiliang Ying

TL;DR
The paper introduces the Attentive Perceptron, a modified perceptron algorithm that uses a focus of attention mechanism to evaluate fewer features on easy examples, significantly speeding up training and prediction with minimal accuracy loss.
Contribution
It presents a novel attention mechanism for perceptrons that reduces feature evaluation, improving efficiency while maintaining accuracy.
Findings
Achieves significant speedups in training and prediction.
Maintains high accuracy with fewer feature evaluations.
Effectively filters easy examples to focus computation on hard ones.
Abstract
We propose a focus of attention mechanism to speed up the Perceptron algorithm. Focus of attention speeds up the Perceptron algorithm by lowering the number of features evaluated throughout training and prediction. Whereas the traditional Perceptron evaluates all the features of each example, the Attentive Perceptron evaluates less features for easy to classify examples, thereby achieving significant speedups and small losses in prediction accuracy. Focus of attention allows the Attentive Perceptron to stop the evaluation of features at any interim point and filter the example. This creates an attentive filter which concentrates computation at examples that are hard to classify, and quickly filters examples that are easy to classify.
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsNeural Networks and Applications · Machine Learning and Data Classification · Domain Adaptation and Few-Shot Learning
