Multiscale Sparse Microcanonical Models

TL;DR

This paper introduces multiscale microcanonical models for approximating non-Gaussian stationary processes with long-range correlations, utilizing wavelet and scattering transforms to capture sparsity and structure.

Contribution

It proposes a novel microcanonical gradient descent method for sampling and approximating complex processes, with theoretical analysis of convergence and entropy control.

Findings

Effective approximation of non-Gaussian processes using multiscale energy vectors.

Demonstrated applications in image and audio texture synthesis.

Analysis of convergence properties of the proposed sampling algorithms.

Abstract

We study approximations of non-Gaussian stationary processes having long range correlations with microcanonical models. These models are conditioned by the empirical value of an energy vector, evaluated on a single realization. Asymptotic properties of maximum entropy microcanonical and macrocanonical processes and their convergence to Gibbs measures are reviewed. We show that the Jacobian of the energy vector controls the entropy rate of microcanonical processes. Sampling maximum entropy processes through MCMC algorithms require too many operations when the number of constraints is large. We define microcanonical gradient descent processes by transporting a maximum entropy measure with a gradient descent algorithm which enforces the energy conditions. Convergence and symmetries are analyzed. Approximations of non-Gaussian processes with long range interactions are defined with multiscale energy vectors computed with wavelet and scattering transforms. Sparsity properties are captured with $\bf l^1$ norms. Approximations of Gaussian, Ising and point processes are studied, as well as image and audio texture synthesis.