Learning SMaLL Predictors

TL;DR

This paper introduces SMaLL, a new machine learning algorithm designed to efficiently train small, resource-constrained predictors inspired by k-DNF Boolean formula learning, with demonstrated empirical benefits.

Contribution

The paper presents the novel SMaLL algorithm for training compact predictors, including its formal derivation and empirical validation.

Findings

SMaLL effectively trains small predictors under resource constraints.

Empirical results show SMaLL outperforms existing methods in relevant tasks.

The approach is inspired by k-DNF Boolean formula learning.

Abstract

We present a new machine learning technique for training small resource-constrained predictors. Our algorithm, the Sparse Multiprototype Linear Learner (SMaLL), is inspired by the classic machine learning problem of learning $k$-DNF Boolean formulae. We present a formal derivation of our algorithm and demonstrate the benefits of our approach with a detailed empirical study.