A Multiplicative Neural Network Architecture: Locality and Regularity of Approximation

TL;DR

This paper introduces a multiplicative neural network architecture that emphasizes multiplicative interactions, demonstrating its superior approximation properties, especially in capturing local irregularities and regularity in functions.

Contribution

The paper establishes a universal approximation theorem for multiplicative neural networks and analyzes their approximation behavior in terms of locality and regularity.

Findings

Residual errors align with regions of reduced regularity

More stable convergence in regularity-sensitive metrics

Enhanced localization and regularity behavior in approximations

Abstract

We introduce a multiplicative neural network architecture in which multiplicative interactions constitute the fundamental representation, rather than appearing as auxiliary components within an additive model. We establish a universal approximation theorem for this architecture and analyze its approximation properties in terms of locality and regularity in Bessel potential spaces. To complement the theoretical results, we conduct numerical experiments on representative targets exhibiting sharp transition layers or pointwise loss of higher-order regularity. The experiments focus on the spatial structure of approximation errors and on regularity-sensitive quantities, in particular, the convergence of Zygmund-type seminorms. The results show that the proposed multiplicative architecture yields residual error structures that are more tightly aligned with regions of reduced regularity and exhibit more stable convergence in regularity-sensitive metrics. These results demonstrate that adopting a multiplicative representation format has concrete implications for the localization and regularity behavior of neural network approximations, providing a direct connection between architectural design and analytical properties of the approximating functions.