Generalized moduli of continuity under irregular or random deformations via multiscale analysis

TL;DR

This paper analyzes the stability of signals under irregular or random deformations using multiscale harmonic analysis, establishing conditions for robustness and bounds on instability in the context of deep neural networks.

Contribution

It introduces a generalized modulus of continuity framework for stability analysis of signals under deformations, extending results to multiresolution spaces and stochastic deformation models.

Findings

Stability in $L^2$ when deformation magnitude is small relative to scale

Sharp bounds on growth rate of instability for large deformations

Extension of stability results to Besov spaces and stochastic models

Abstract

Motivated by the problem of robustness to deformations of the input for deep convolutional neural networks, we identify signal classes which are inherently stable to irregular deformations induced by distortion fields $\tau\in L^\infty(\mathbb{R}^d;\mathbb{R}^d)$, to be characterized in terms of a generalized modulus of continuity associated with the deformation operator. Resorting to ideas of harmonic and multiscale analysis, we prove that for signals in multiresolution approximation spaces $U_s$ at scale $s$, stability in $L^2$ holds in the regime $\|\tau\|_{L^\infty}/s\ll 1$ - essentially as an effect of the uncertainty principle. Instability occurs when $\|\tau\|_{L^\infty}/s\gg 1$, and we provide a sharp upper bound for the asymptotic growth rate. The stability results are then extended to signals in the Besov space $B^{d/2}_{2,1}$ tailored to the given multiresolution approximation. We also consider the case of more general time-frequency deformations. Finally, we provide stochastic versions of the aforementioned results, namely we study the issue of stability in mean when $\tau(x)$ is modeled as a random field (not bounded, in general) with identically distributed variables $|\tau(x)|$, $x\in\mathbb{R}^d$.