In-Distribution Interpretability for Challenging Modalities

TL;DR

This paper explores a flexible interpretability framework using generative models to understand complex modalities like music and urban environment simulations, advancing the transparency of deep neural networks.

Contribution

It demonstrates the application of a generative-model-based interpretability method to diverse challenging modalities beyond traditional data types.

Findings

Effective interpretation of music models

Insights into physical urban environment simulations

Framework adaptable to various complex modalities

Abstract

It is widely recognized that the predictions of deep neural networks are difficult to parse relative to simpler approaches. However, the development of methods to investigate the mode of operation of such models has advanced rapidly in the past few years. Recent work introduced an intuitive framework which utilizes generative models to improve on the meaningfulness of such explanations. In this work, we display the flexibility of this method to interpret diverse and challenging modalities: music and physical simulations of urban environments.