Benchmarking deep inverse models over time, and the neural-adjoint method

TL;DR

This paper benchmarks various deep learning-based inverse modeling methods over time across multiple tasks, introducing the neural-adjoint approach that combines forward model approximation with backpropagation for improved inverse solutions.

Contribution

It introduces a time-based evaluation metric for inverse models and proposes the neural-adjoint method, which outperforms existing approaches in several benchmark tasks.

Findings

Neural-adjoint achieves superior performance in inverse problems.

Time-based evaluation reveals differences in method efficiency.

Benchmarking across diverse tasks demonstrates method robustness.

Abstract

We consider the task of solving generic inverse problems, where one wishes to determine the hidden parameters of a natural system that will give rise to a particular set of measurements. Recently many new approaches based upon deep learning have arisen generating impressive results. We conceptualize these models as different schemes for efficiently, but randomly, exploring the space of possible inverse solutions. As a result, the accuracy of each approach should be evaluated as a function of time rather than a single estimated solution, as is often done now. Using this metric, we compare several state-of-the-art inverse modeling approaches on four benchmark tasks: two existing tasks, one simple task for visualization and one new task from metamaterial design. Finally, inspired by our conception of the inverse problem, we explore a solution that uses a deep learning model to approximate the forward model, and then uses backpropagation to search for good inverse solutions. This approach, termed the neural-adjoint, achieves the best performance in many scenarios.