On the role of memorization in learned priors for geophysical inverse problems

TL;DR

This paper investigates how memorization in deep generative models affects learned priors for seismic inversion, revealing that memorized models effectively act as likelihood-weighted combinations of training examples, impacting posterior sampling.

Contribution

It provides a theoretical analysis of memorization effects in learned priors, especially diffusion models, and demonstrates their impact on seismic inverse problem solutions.

Findings

Memorized priors reduce to reweighted empirical distributions.

Diffusion models produce Gaussian mixture priors in closed form.

Memorization influences the shape and uncertainty of the posterior.

Abstract

Learned priors based on deep generative models offer data-driven regularization for seismic inversion, but training them requires a dataset of representative subsurface models -- a resource that is inherently scarce in geoscience applications. Since the training objective of most generative models can be cast as maximum likelihood on a finite dataset, any such model risks converging to the empirical distribution -- effectively memorizing the training examples rather than learning the underlying geological distribution. We show that the posterior under such a memorized prior reduces to a reweighted empirical distribution -- i.e., a likelihood-weighted lookup among the stored training examples. For diffusion models specifically, memorization yields a Gaussian mixture prior in closed form, and linearizing the forward operator around each training example gives a Gaussian mixture posterior whose components have widths and shifts governed by the local Jacobian. We validate these predictions on a stylized inverse problem and demonstrate the consequences of memorization through diffusion posterior sampling for full waveform inversion.