Universal approximations of permutation invariant/equivariant functions by deep neural networks

TL;DR

This paper develops a theoretical framework showing that deep neural networks can universally approximate permutation invariant and equivariant functions with fewer parameters by incorporating group actions into their layers.

Contribution

It introduces a method to construct neural network approximators for G-invariant/equivariant functions that are more parameter-efficient using representation theory.

Findings

Neural networks can universally approximate G-invariant/equivariant functions.

The proposed models have exponentially fewer parameters than traditional approaches.

Representation theory underpins the construction of these efficient neural network approximators.

Abstract

In this paper, we develop a theory about the relationship between $G$-invariant/equivariant functions and deep neural networks for finite group $G$. Especially, for a given $G$-invariant/equivariant function, we construct its universal approximator by deep neural network whose layers equip $G$-actions and each affine transformations are $G$-equivariant/invariant. Due to representation theory, we can show that this approximator has exponentially fewer free parameters than usual models.