SOInter: A Novel Deep Energy Based Interpretation Method for Explaining Structured Output Models

TL;DR

This paper introduces SOInter, a deep energy-based interpretation method that explains structured output models by considering correlations among output variables, improving interpretability for complex models.

Contribution

The paper presents a novel energy-based interpretation technique that accounts for output variable correlations, enhancing explanation accuracy for structured output models.

Findings

Effective in explaining complex structured models

Improves explanation accuracy by considering output correlations

Validated on simulated and real datasets

Abstract

We propose a novel interpretation technique to explain the behavior of structured output models, which learn mappings between an input vector to a set of output variables simultaneously. Because of the complex relationship between the computational path of output variables in structured models, a feature can affect the value of output through other ones. We focus on one of the outputs as the target and try to find the most important features utilized by the structured model to decide on the target in each locality of the input space. In this paper, we assume an arbitrary structured output model is available as a black box and argue how considering the correlations between output variables can improve the explanation performance. The goal is to train a function as an interpreter for the target output variable over the input space. We introduce an energy-based training process for the interpreter function, which effectively considers the structural information incorporated into the model to be explained. The effectiveness of the proposed method is confirmed using a variety of simulated and real data sets.