Harmonic Decompositions of Convolutional Networks

TL;DR

This paper characterizes the function space of convolutional networks using harmonic analysis and reproducing kernel Hilbert spaces, revealing a decomposition into elementary functions and establishing bounds on approximation and estimation errors.

Contribution

It introduces a novel harmonic decomposition framework for convolutional networks and connects it to statistical properties and error trade-offs.

Findings

Convolutional networks can be decomposed into elementary harmonic functions.

The functional analysis relates to the ANOVA decomposition in statistics.

Statistical bounds highlight the trade-off between approximation and estimation errors.

Abstract

We present a description of the function space and the smoothness class associated with a convolutional network using the machinery of reproducing kernel Hilbert spaces. We show that the mapping associated with a convolutional network expands into a sum involving elementary functions akin to spherical harmonics. This functional decomposition can be related to the functional ANOVA decomposition in nonparametric statistics. Building off our functional characterization of convolutional networks, we obtain statistical bounds highlighting an interesting trade-off between the approximation error and the estimation error.