Neural Oscillators are Universal
Samuel Lanthaler, T. Konstantin Rusch, Siddhartha Mishra

TL;DR
This paper proves that neural oscillators are a universal class capable of approximating any continuous causal operator between time-varying functions, providing theoretical support for their use in machine learning architectures.
Contribution
It introduces an abstract class of neural oscillators and proves their universality in approximating continuous causal operators, justifying their application in ML systems.
Findings
Neural oscillators can approximate any continuous causal operator.
A combination of forced harmonic oscillators with nonlinear read-out suffices.
Theoretical foundation for oscillator-based machine learning architectures.
Abstract
Coupled oscillators are being increasingly used as the basis of machine learning (ML) architectures, for instance in sequence modeling, graph representation learning and in physical neural networks that are used in analog ML devices. We introduce an abstract class of neural oscillators that encompasses these architectures and prove that neural oscillators are universal, i.e, they can approximate any continuous and casual operator mapping between time-varying functions, to desired accuracy. This universality result provides theoretical justification for the use of oscillator based ML systems. The proof builds on a fundamental result of independent interest, which shows that a combination of forced harmonic oscillators with a nonlinear read-out suffices to approximate the underlying operators.
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
Taxonomy
TopicsNeural Networks and Applications · Model Reduction and Neural Networks · Control Systems and Identification
