Distributional Sensitivity Analysis: Enabling Differentiability in Sample-Based Inference

TL;DR

This paper introduces two analytical formulas and four numerical algorithms to estimate the sensitivity of sample-based distributions, enabling differentiability and uncertainty quantification in complex inverse problems without model fitting.

Contribution

It provides novel formulas and algorithms for sensitivity analysis that work with black-box samplers, enhancing differentiability and inference in high-dimensional, sample-based settings.

Findings

Validated the correctness of numerical algorithms.

Demonstrated effectiveness in a nuclear physics application.

Enabled differentiability of arbitrary sampling routines.

Abstract

We present two analytical formulae for estimating the sensitivity -- namely, the gradient or Jacobian -- at given realizations of an arbitrary-dimensional random vector with respect to its distributional parameters. The first formula interprets this sensitivity as partial derivatives of the inverse mapping associated with the vector of 1-D conditional distributions. The second formula, intended for optimization methods that tolerate inexact gradients, introduces a diagonal approximation that reduces computational cost at the cost of some accuracy. We additionally provide four second-order numerical algorithms to approximate both formulae when closed forms are unavailable. We performed verification and validation studies to demonstrate the correctness of these numerical algorithms and the effectiveness of the proposed formulae. A nuclear physics application showcases how our work enables uncertainty quantification and parameter inference for quantum correlation functions. Our approach differs from existing methods by avoiding the need for model fitting, knowledge of sampling algorithms, and evaluation of high-dimensional integrals. It is therefore particularly useful for sample-based inverse problems when the sampler operates as a black box or requires expensive physics simulations. Moreover, our method renders arbitrary sampling subroutines differentiable, facilitating their integration into programming frameworks for deep learning and automatic differentiation. Algorithmic details and code implementations are provided in this paper and in our open-source software DistroSA to enable reproducibility and further development.