Frequency-adaptive tensor neural networks for high-dimensional multi-scale problems

TL;DR

This paper introduces a frequency-adaptive tensor neural network approach that enhances high-dimensional multi-scale problem solving by Fourier analysis and random Fourier features, overcoming frequency limitations.

Contribution

The paper proposes a novel frequency-adaptive TNN method using Fourier features and tensor structure to improve high-frequency feature capture in high-dimensional problems.

Findings

Enhanced ability of TNNs to capture high-frequency features.

Significant improvement in solving multi-scale high-dimensional problems.

Validated robustness through extensive numerical experiments.

Abstract

Tensor neural networks (TNNs) have demonstrated their superiority in solving high-dimensional problems. However, similar to conventional neural networks, TNNs are also influenced by the Frequency Principle, which limits their ability to accurately capture high-frequency features of the solution. In this work, we analyze the training dynamics of TNNs by Fourier analysis and enhance their expressivity for high-dimensional multi-scale problems by incorporating random Fourier features. Leveraging the inherent tensor structure of TNNs, we further propose a novel approach to extract frequency features of high-dimensional functions by performing the Discrete Fourier Transform to one-dimensional component functions. This strategy effectively mitigates the curse of dimensionality. Building on this idea, we propose a frequency-adaptive TNNs algorithm, which significantly improves the ability of TNNs in solving complex multi-scale problems. Extensive numerical experiments are performed to validate the effectiveness and robustness of the proposed frequency-adaptive TNNs algorithm.