Optimization Landscapes Learned: Proxy Networks Boost Convergence in Physics-based Inverse Problems

TL;DR

This paper demonstrates that proxy neural networks can learn complex physics-based inverse problem landscapes, and their regularization improves convergence over traditional optimization methods.

Contribution

It introduces a method to learn and regularize complex inverse problem landscapes using neural networks, enhancing optimization convergence.

Findings

Proxy networks replicate complex loss landscapes.

Regularization controls landscape complexity.

Improved convergence over BFGS.

Abstract

Solving inverse problems in physics is central to understanding complex systems and advancing technologies in various fields. Iterative optimization algorithms, commonly used to solve these problems, often encounter local minima, chaos, or regions with zero gradients. This is due to their overreliance on local information and highly chaotic inverse loss landscapes governed by underlying partial differential equations (PDEs). In this work, we show that deep neural networks successfully replicate such complex loss landscapes through spatio-temporal trajectory inputs. They also offer the potential to control the underlying complexity of these chaotic loss landscapes during training through various regularization methods. We show that optimizing on network-smoothened loss landscapes leads to improved convergence in predicting optimum inverse parameters over conventional momentum-based optimizers such as BFGS on multiple challenging problems.