On the effectiveness of neural priors in modeling dynamical systems

TL;DR

This paper explores how simple coordinate neural networks can effectively model dynamical systems without explicit regularization, providing insights into their architectural regularization benefits from a signal processing perspective.

Contribution

It introduces a signal processing interpretation of coordinate networks for dynamical systems and demonstrates their effectiveness without explicit regularizers.

Findings

Simple coordinate networks can model dynamical systems effectively.

Architectural regularization in neural networks can be understood through signal processing.

Few-layer coordinate networks suffice for multiple dynamical system modeling tasks.

Abstract

Modelling dynamical systems is an integral component for understanding the natural world. To this end, neural networks are becoming an increasingly popular candidate owing to their ability to learn complex functions from large amounts of data. Despite this recent progress, there has not been an adequate discussion on the architectural regularization that neural networks offer when learning such systems, hindering their efficient usage. In this paper, we initiate a discussion in this direction using coordinate networks as a test bed. We interpret dynamical systems and coordinate networks from a signal processing lens, and show that simple coordinate networks with few layers can be used to solve multiple problems in modelling dynamical systems, without any explicit regularizers.