Neural Functions for Learning Periodic Signal
Woojin Cho, Minju Jo, Kookjin Lee, Noseong Park
TL;DR
This paper introduces a neural network architecture designed to better learn and extrapolate periodic signals, addressing overfitting issues in traditional coordinate-based MLPs, and demonstrating improved performance on differential equations and time series tasks.
Contribution
The paper proposes a novel neural network architecture that explicitly captures periodic patterns to enhance generalization and extrapolation of periodic signals.
Findings
Improved extrapolation in differential equation solutions
Enhanced time series forecasting accuracy
Better generalization to unseen data regions
Abstract
As function approximators, deep neural networks have served as an effective tool to represent various signal types. Recent approaches utilize multi-layer perceptrons (MLPs) to learn a nonlinear mapping from a coordinate to its corresponding signal, facilitating the learning of continuous neural representations from discrete data points. Despite notable successes in learning diverse signal types, coordinate-based MLPs often face issues of overfitting and limited generalizability beyond the training region, resulting in subpar extrapolation performance. This study addresses scenarios where the underlying true signals exhibit periodic properties, either spatially or temporally. We propose a novel network architecture, which extracts periodic patterns from measurements and leverages this information to represent the signal, thereby enhancing generalization and improving extrapolation…
Peer Reviews
Decision·ICLR 2025 Poster
**Originality**: The NeRT framework models periodic signals by effectively separating periodic and scale factors. This method creatively combines concepts from Fourier analysis and neural network architectures, leading to a more nuanced understanding of periodic behaviors in data. **Clarity**: The paper is almost well-written and clear. The authors explain the motivation for their approach, the architecture of NeRT, and the experimental results. Diagrams and examples support key points througho
1. Although the proposed method can model/forecast periodic signals, I am not convinced it outperforms state-of-the-art RNNs [2] or other successful methods for analyzing and modeling time series data, such as Koopman-based methods [1]. These methods can model and forecast not only periodic time series but also more challenging time series data related to chaotic systems, see for instance [1,2,3,4,5]. - Could the authors compare more aspects, for example, **training time** and the **hyperparame
- The paper introduces a novel architecture for implicit neural representation, consisting of two separate internal networks to learn the periodic and scale components of an observed signal. - The proposed approach enables forecasting and imputation using a single trained model. - Experiments show that the framework achieves better performance than baselines across multiple tasks and datasets. - The paper is well written and easy to follow.
- Could the authors provide insights into why an INR-based approach excels in forecasting and imputation of time series compared to RNN-style approaches that use time series sequences as input? Is the superior performance primarily due to the ability of the proposed method to better capture periodicity present in the datasets? - The paper currently lacks comparisons with state-of-the-art approaches for the imputation/interpolation task. - While the paper claims that the proposed approach can mod
This paper is very well-written with extensive experimental study to demonstrate the proposed method (NeRT). The method is simple to understand and clear to interpret. The Fourier mapping layer can be very helpful to model this kind of periodic behavior from time-series. The experimental section is also very strong, which outperforms all baseline methods. The detailed visualization and ablation study further show how NeRT is a very effective strategy for complex signals with periodic behaviors.
The weaknesses of this paper are also clear which is mainly coming from the fact that NeRT could only potentially able to model periodic signals. The Fourier embedding/feature may be a huge constraint since the signal must have a fixed period. Yet this is understandable since there are many natural phenomena that has fixed frequency (e.g. the global temperature, traffic). There are also some issues in terms of presentation that may potentially need to be fixed.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsModel Reduction and Neural Networks · Time Series Analysis and Forecasting · Neural Networks and Reservoir Computing