Time Series Continuous Modeling for Imputation and Forecasting with Implicit Neural Representations
Etienne Le Naour, Louis Serrano, L\'eon Migus, Yuan Yin, Ghislain Agoua, Nicolas Baskiotis, Patrick Gallinari, Vincent Guigue

TL;DR
This paper presents a continuous-time neural modeling approach for time series imputation and forecasting, effectively handling irregular, missing, or unaligned data, and achieving state-of-the-art results.
Contribution
It introduces a novel implicit neural representation-based model with meta-learning adaptation for improved time series imputation and forecasting.
Findings
Achieves state-of-the-art performance on classical benchmarks.
Effectively handles irregular and missing data scenarios.
Outperforms alternative continuous-time models.
Abstract
We introduce a novel modeling approach for time series imputation and forecasting, tailored to address the challenges often encountered in real-world data, such as irregular samples, missing data, or unaligned measurements from multiple sensors. Our method relies on a continuous-time-dependent model of the series' evolution dynamics. It leverages adaptations of conditional, implicit neural representations for sequential data. A modulation mechanism, driven by a meta-learning algorithm, allows adaptation to unseen samples and extrapolation beyond observed time-windows for long-term predictions. The model provides a highly flexible and unified framework for imputation and forecasting tasks across a wide range of challenging scenarios. It achieves state-of-the-art performance on classical benchmarks and outperforms alternative time-continuous models.
Peer Reviews
Decision·Submitted to ICLR 2024
Since the method is a pretty straightforward application of Dupont et al. (2022) to time series, the approach is sound. The results on imputation are decent. The methods shows promising results in terms of MAE and the imputation in Figure 3 looks good compared to BRITS. However, it seems there is still a lot of performance improvement left on the table. Electricity is a pretty simple periodic dataset so I can imagine achieving better results with further tuning or some other model. Forecasting
The approach has limited novelty since it's mostly building upon known previous work. This same architecture can be applied to images, point clouds, and so on. Although the discussion of implementation choices is a nice addition, they are again not necessarily time series dependent. Results on imputation are decent, but the method is not beating other baselines most of the time. It is usually close to BRITS and some other baselines. This might indicate used datasets are too simple. Also, using
- The paper is written clearly, elaborating the architecture design, and training/test algorithms. - Time-series modeling has not been investigated much in the INR literature and this paper provides some insights that modeling time-series data in a continuous neural function can be beneficial. - The paper compares the proposed method with several important baselines.
- Although the domain of the application (i.e., time-series modeling) is new and the proposed design brings an idea of the latent state evolution (in forecasting), the novelty seems to be limited. The overall architectural design follows the FFN architecture (Tancik, et al, 2020) without any consideration on how to handle multivariate time-series. Also, the idea of the latent modulation and the meta-learning-based training algorithm largely follow the previous approach (Dupont, et al, 2022). Fin
Excellent explanation of the method and figures diagramming what was done and the distinctions between training and inference periods, for both the imputation and the forecasting applications. The algorithm's ability to be applied to previously unseen datasets/time series is a definite strength.
The conclusion/discussion was quite brief. I would have loved to read more about limitations and the approach for extending this to the multivariate case. Section 3.4 was a strong inclusion of rationale for their authors' choices, but the disjoint list of conclusions and redirection to the Appendix was weak. Perhaps some (unnecessary) details of the datasets could be left to the appendix to provide space for more description of the actual method.
Code & Models
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsGaussian Processes and Bayesian Inference · Time Series Analysis and Forecasting · Domain Adaptation and Few-Shot Learning
