Central limit theorems for the outputs of fully convolutional neural networks with time series input

TL;DR

This paper proves that the outputs of fully convolutional neural networks with global average pooling, when applied to short-range dependent linear process time series, are asymptotically Gaussian as series length increases.

Contribution

It provides the first theoretical proof of Gaussian limits for FCN outputs on time series and introduces a learnable weighted pooling extension.

Findings

Outputs are asymptotically Gaussian for large series length.

The proof uses existing time series theoretical tools.

Proposes a learnable weighted pooling layer.

Abstract

Deep learning is widely deployed for time series learning tasks such as classification and forecasting. Despite the empirical successes, only little theory has been developed so far in the time series context. In this work, we prove that if the network inputs are generated from short-range dependent linear processes, the outputs of fully convolutional neural networks (FCNs) with global average pooling (GAP) are asymptotically Gaussian and the limit is attained if the length of the observed time series tends to infinity. The proof leverages existing tools from the theoretical time series literature. Based on our theory, we propose a generalization of the GAP layer by considering a global weighted pooling step with slowly varying, learnable coefficients.