Latent Concept-based Explanation of NLP Models

TL;DR

This paper introduces LACOAT, a method that explains NLP model predictions by mapping input words into a latent space to reveal context-dependent facets, offering more informative insights than traditional word-based explanations.

Contribution

The paper proposes a novel latent concept-based explanation method for NLP models, addressing limitations of word-based explanations by capturing context-dependent facets of words.

Findings

LACOAT provides more nuanced explanations of model predictions.

The method effectively captures multiple facets of words based on context.

It enhances interpretability of NLP models through latent space analysis.

Abstract

Interpreting and understanding the predictions made by deep learning models poses a formidable challenge due to their inherently opaque nature. Many previous efforts aimed at explaining these predictions rely on input features, specifically, the words within NLP models. However, such explanations are often less informative due to the discrete nature of these words and their lack of contextual verbosity. To address this limitation, we introduce the Latent Concept Attribution method (LACOAT), which generates explanations for predictions based on latent concepts. Our foundational intuition is that a word can exhibit multiple facets, contingent upon the context in which it is used. Therefore, given a word in context, the latent space derived from our training process reflects a specific facet of that word. LACOAT functions by mapping the representations of salient input words into the training latent space, allowing it to provide latent context-based explanations of the prediction.