Central and Non-central Limit Theorems arising from the Scattering Transform and its Neural Activation Generalization

TL;DR

This paper introduces a generalized scattering transform called NAST that incorporates neural activation functions, analyzing its statistical properties and limit theorems for processing complex, non-stationary time series.

Contribution

It extends the scattering transform framework to include neural activations, providing new CLT and non-CLT results for Gaussian processes and demonstrating its effectiveness in analyzing non-stationary data.

Findings

NAST exhibits non-expansion and translational invariance.

Statistical properties depend on activation functions and filters.

Numerical simulations validate theoretical results.

Abstract

Motivated by analyzing complicated and non-stationary time series, we study a generalization of the scattering transform (ST) that includes broad neural activation functions, which is called neural activation ST (NAST). On the whole, NAST is a transform that comprises a sequence of ``neural processing units'', each of which applies a high pass filter to the input from the previous layer followed by a composition with a nonlinear function as the output to the next neuron. Here, the nonlinear function models how a neuron gets excited by the input signal. In addition to showing properties like non-expansion, horizontal translational invariability and insensitivity to local deformation, the statistical properties of the second order NAST of a Gaussian process with various dependence and (non-)stationarity structure and its interaction with the chosen high pass filters and activation functions are explored and central limit theorem (CLT) and non-CLT results are provided. Numerical simulations are also provided. The results explain how NAST processes complicated and non-stationary time series, and pave a way towards statistical inference based on NAST under the non-null case.