LS-Tree: Model Interpretation When the Data Are Linguistic

TL;DR

This paper introduces LS-Tree, a method for interpreting linguistic data models by assigning importance scores to words based on parse trees and syntactic structures, improving interpretability and interaction detection.

Contribution

It proposes a novel importance scoring method using least-squares and syntactic trees, with an axiomatic foundation linked to coalitional game theory.

Findings

Effectively interprets language models using parse-tree importance scores

Detects and quantifies word interactions in sentences

Enhances interpretability and diagnostics for NLP models

Abstract

We study the problem of interpreting trained classification models in the setting of linguistic data sets. Leveraging a parse tree, we propose to assign least-squares based importance scores to each word of an instance by exploiting syntactic constituency structure. We establish an axiomatic characterization of these importance scores by relating them to the Banzhaf value in coalitional game theory. Based on these importance scores, we develop a principled method for detecting and quantifying interactions between words in a sentence. We demonstrate that the proposed method can aid in interpretability and diagnostics for several widely-used language models.