Exploring and Interacting with the Set of Good Sparse Generalized Additive Models

TL;DR

This paper introduces algorithms to efficiently approximate the set of all near-optimal sparse generalized additive models, enabling better exploration, interpretation, and customization of models in practical applications.

Contribution

It presents novel algorithms for approximating the Rashomon set of sparse generalized additive models using ellipsoids, facilitating practical model exploration and interaction.

Findings

High fidelity approximation of the Rashomon set

Effective in variable importance analysis

Supports practical model customization

Abstract

In real applications, interaction between machine learning models and domain experts is critical; however, the classical machine learning paradigm that usually produces only a single model does not facilitate such interaction. Approximating and exploring the Rashomon set, i.e., the set of all near-optimal models, addresses this practical challenge by providing the user with a searchable space containing a diverse set of models from which domain experts can choose. We present algorithms to efficiently and accurately approximate the Rashomon set of sparse, generalized additive models with ellipsoids for fixed support sets and use these ellipsoids to approximate Rashomon sets for many different support sets. The approximated Rashomon set serves as a cornerstone to solve practical challenges such as (1) studying the variable importance for the model class; (2) finding models under user-specified constraints (monotonicity, direct editing); and (3) investigating sudden changes in the shape functions. Experiments demonstrate the fidelity of the approximated Rashomon set and its effectiveness in solving practical challenges.