Neural Oscillators are Universal

TL;DR

This paper proves that neural oscillators are a universal class capable of approximating any continuous causal operator between time-varying functions, providing theoretical support for their use in machine learning architectures.

Contribution

It introduces an abstract class of neural oscillators and proves their universality in approximating continuous causal operators, justifying their application in ML systems.

Findings

Neural oscillators can approximate any continuous causal operator.

A combination of forced harmonic oscillators with nonlinear read-out suffices.

Theoretical foundation for oscillator-based machine learning architectures.

Abstract

Coupled oscillators are being increasingly used as the basis of machine learning (ML) architectures, for instance in sequence modeling, graph representation learning and in physical neural networks that are used in analog ML devices. We introduce an abstract class of neural oscillators that encompasses these architectures and prove that neural oscillators are universal, i.e, they can approximate any continuous and casual operator mapping between time-varying functions, to desired accuracy. This universality result provides theoretical justification for the use of oscillator based ML systems. The proof builds on a fundamental result of independent interest, which shows that a combination of forced harmonic oscillators with a nonlinear read-out suffices to approximate the underlying operators.