Feature Importance Estimation with Self-Attention Networks

TL;DR

This paper investigates using self-attention networks to estimate feature importance in tabular data, comparing their effectiveness with traditional methods and demonstrating their ability to identify relevant features and interactions.

Contribution

It introduces a novel SAN-based approach for feature importance estimation and provides the first scale-free comparison with established methods across multiple datasets.

Findings

SANs identify similar high-ranked features as traditional methods

SANs can detect larger feature interactions relevant for prediction

SANs sometimes outperform baselines in predictive accuracy

Abstract

Black-box neural network models are widely used in industry and science, yet are hard to understand and interpret. Recently, the attention mechanism was introduced, offering insights into the inner workings of neural language models. This paper explores the use of attention-based neural networks mechanism for estimating feature importance, as means for explaining the models learned from propositional (tabular) data. Feature importance estimates, assessed by the proposed Self-Attention Network (SAN) architecture, are compared with the established ReliefF, Mutual Information and Random Forest-based estimates, which are widely used in practice for model interpretation. For the first time we conduct scale-free comparisons of feature importance estimates across algorithms on ten real and synthetic data sets to study the similarities and differences of the resulting feature importance estimates, showing that SANs identify similar high-ranked features as the other methods. We demonstrate that SANs identify feature interactions which in some cases yield better predictive performance than the baselines, suggesting that attention extends beyond interactions of just a few key features and detects larger feature subsets relevant for the considered learning task.