xInv: Explainable Optimization of Inverse Problems

TL;DR

This paper introduces xInv, a method that generates human-interpretable explanations for inverse problem optimization processes by instrumenting differentiable simulators and using language models to produce natural language summaries.

Contribution

The paper presents a novel approach to explain inverse problem optimization by integrating natural language explanations derived from simulator events, enhancing interpretability for domain experts.

Findings

Effective explanation generation demonstrated on illustrative problems

Natural language summaries improve understanding of optimization processes

Method bridges the gap between complex optimization traces and human interpretability

Abstract

Inverse problems are central to a wide range of fields, including healthcare, climate science, and agriculture. They involve the estimation of inputs, typically via iterative optimization, to some known forward model so that it produces a desired outcome. Despite considerable development in the explainability and interpretability of forward models, the iterative optimization of inverse problems remains largely cryptic to domain experts. We propose a methodology to produce explanations, from traces produced by an optimizer, that are interpretable by humans at the abstraction of the domain. The central idea in our approach is to instrument a differentiable simulator so that it emits natural language events during its forward and backward passes. In a post-process, we use a Language Model to create an explanation from the list of events. We demonstrate the effectiveness of our approach with an illustrative optimization problem and an example involving the training of a neural network.