Initial Model Incorporation for Deep Learning FWI: Pretraining or Denormalization?

TL;DR

This paper compares pretraining and denormalization methods for incorporating initial models into neural network-based full waveform inversion, finding denormalization more effective and simpler.

Contribution

It systematically evaluates the influence of pretraining versus denormalization in neural network reparameterized FWI, highlighting denormalization's advantages.

Findings

Denormalization simplifies workflows and accelerates convergence.

Pretraining can cause network parameters to become inactive.

Denormalization improves inversion accuracy.

Abstract

Subsurface property neural network reparameterized full waveform inversion (FWI) has emerged as an effective unsupervised learning framework, which can invert stably with an inaccurate starting model. It updates the trainable neural network parameters instead of fine-tuning on the subsurface model directly. There are primarily two ways to embed the prior knowledge of the initial model into neural networks, that is, pretraining and denormalization. Pretraining first regulates the neural networks' parameters by fitting the initial velocity model; Denormalization directly adds the outputs of the network into the initial models without pretraining. In this letter, we systematically investigate the influence of the two ways of initial model incorporation for the neural network reparameterized FWI. We demonstrate that pretraining requires inverting the model perturbation based on a constant velocity value (mean) with a two-stage implementation. It leads to a complex workflow and inconsistency of objective functions in the two-stage process, causing the network parameters to become inactive and lose plasticity. Experimental results demonstrate that denormalization can simplify workflows, accelerate convergence, and enhance inversion accuracy compared with pretraining.