Joint Group Invariant Functions on Data-Parameter Domain Induce Universal Neural Networks

TL;DR

This paper introduces a group-theoretic framework for neural networks, demonstrating their universality through the ridgelet transform and symmetry considerations, bridging geometric deep learning and harmonic analysis.

Contribution

It provides a systematic method to induce neural networks from joint group invariant functions and offers a unified, group-theoretic proof of their universality.

Findings

The ridgelet transform describes parameter arrangements for target functions.

The approach generalizes to various network types including deep and dual voice transforms.

It connects geometric deep learning with abstract harmonic analysis.

Abstract

The symmetry and geometry of input data are considered to be encoded in the internal data representation inside the neural network, but the specific encoding rule has been less investigated. In this study, we present a systematic method to induce a generalized neural network and its right inverse operator, called the ridgelet transform, from a joint group invariant function on the data-parameter domain. Since the ridgelet transform is an inverse, (1) it can describe the arrangement of parameters for the network to represent a target function, which is understood as the encoding rule, and (2) it implies the universality of the network. Based on the group representation theory, we present a new simple proof of the universality by using Schur's lemma in a unified manner covering a wide class of networks, for example, the original ridgelet transform, formal deep networks, and the dual voice transform. Since traditional universality theorems were demonstrated based on functional analysis, this study sheds light on the group theoretic aspect of the approximation theory, connecting geometric deep learning to abstract harmonic analysis.