The Most Important Features in Generalized Additive Models Might Be Groups of Features

TL;DR

This paper presents a new method for assessing the importance of feature groups in Generalized Additive Models, highlighting the significance of joint effects over individual features, especially in high-dimensional and multimodal datasets.

Contribution

The paper introduces an efficient, model-agnostic approach to evaluate group importance in GAMs, allowing posthoc, overlapping, and high-dimensional group analysis without retraining.

Findings

Group importance analysis reveals more accurate insights in medical datasets.

Synthetic experiments demonstrate the method's behavior across data regimes.

Application to neuroscience and health data shows holistic understanding of features.

Abstract

While analyzing the importance of features has become ubiquitous in interpretable machine learning, the joint signal from a group of related features is sometimes overlooked or inadvertently excluded. Neglecting the joint signal could bypass a critical insight: in many instances, the most significant predictors are not isolated features, but rather the combined effect of groups of features. This can be especially problematic for datasets that contain natural groupings of features, including multimodal datasets. This paper introduces a novel approach to determine the importance of a group of features for Generalized Additive Models (GAMs) that is efficient, requires no model retraining, allows defining groups posthoc, permits overlapping groups, and remains meaningful in high-dimensional settings. Moreover, this definition offers a parallel with explained variation in statistics. We showcase properties of our method on three synthetic experiments that illustrate the behavior of group importance across various data regimes. We then demonstrate the importance of groups of features in identifying depressive symptoms from a multimodal neuroscience dataset, and study the importance of social determinants of health after total hip arthroplasty. These two case studies reveal that analyzing group importance offers a more accurate, holistic view of the medical issues compared to a single-feature analysis.