Fourier-domain Variational Formulation and Its Well-posedness for Supervised Learning

TL;DR

This paper introduces a Fourier-domain variational approach for supervised learning, establishing its well-posedness and providing a practical neural network implementation that naturally satisfies the theoretical conditions.

Contribution

It proposes a novel Fourier-domain variational formulation for supervised learning and proves its well-posedness under a critical exponent related to data dimension.

Findings

The formulation circumvents constraints on isolated data points.

A necessary and sufficient condition for well-posedness is established.

Neural networks can effectively implement the formulation, ensuring well-posedness.

Abstract

A supervised learning problem is to find a function in a hypothesis function space given values on isolated data points. Inspired by the frequency principle in neural networks, we propose a Fourier-domain variational formulation for supervised learning problem. This formulation circumvents the difficulty of imposing the constraints of given values on isolated data points in continuum modelling. Under a necessary and sufficient condition within our unified framework, we establish the well-posedness of the Fourier-domain variational problem, by showing a critical exponent depending on the data dimension. In practice, a neural network can be a convenient way to implement our formulation, which automatically satisfies the well-posedness condition.