Exclusion and Inclusion -- A model agnostic approach to feature importance in DNNs

TL;DR

This paper introduces a model-agnostic, robust method for determining phrase-wise feature importance in DNNs, applicable across NLP tasks like regression and classification, addressing the black box challenge.

Contribution

The authors propose a novel, generalizable algorithm for feature importance that is model-agnostic and robust to outliers in NLP applications.

Findings

Effective in both regression and classification tasks

Robust to outliers, capturing essential input features

Applicable across diverse NLP models

Abstract

Deep Neural Networks in NLP have enabled systems to learn complex non-linear relationships. One of the major bottlenecks towards being able to use DNNs for real world applications is their characterization as black boxes. To solve this problem, we introduce a model agnostic algorithm which calculates phrase-wise importance of input features. We contend that our method is generalizable to a diverse set of tasks, by carrying out experiments for both Regression and Classification. We also observe that our approach is robust to outliers, implying that it only captures the essential aspects of the input.