SWAG: A Wrapper Method for Sparse Learning

TL;DR

SWAG is a new wrapper method that creates a library of sparse, interpretable models with similar predictive performance, enhancing interpretability and replicability in machine learning applications.

Contribution

The paper introduces SWAG, a novel wrapper algorithm that efficiently explores attribute space to produce diverse, sparse, and interpretable models with comparable accuracy.

Findings

Produces a low-dimensional, interpretable attribute network

Generates a library of models with equivalent predictive power

Reduces data collection and storage costs

Abstract

The majority of machine learning methods and algorithms give high priority to prediction performance which may not always correspond to the priority of the users. In many cases, practitioners and researchers in different fields, going from engineering to genetics, require interpretability and replicability of the results especially in settings where, for example, not all attributes may be available to them. As a consequence, there is the need to make the outputs of machine learning algorithms more interpretable and to deliver a library of "equivalent" learners (in terms of prediction performance) that users can select based on attribute availability in order to test and/or make use of these learners for predictive/diagnostic purposes. To address these needs, we propose to study a procedure that combines screening and wrapper approaches which, based on a user-specified learning method, greedily explores the attribute space to find a library of sparse learners with consequent low data collection and storage costs. This new method (i) delivers a low-dimensional network of attributes that can be easily interpreted and (ii) increases the potential replicability of results based on the diversity of attribute combinations defining strong learners with equivalent predictive power. We call this algorithm "Sparse Wrapper AlGorithm" (SWAG).