JANET: Joint Adaptive predictioN-region Estimation for Time-series
Eshant English, Eliot Wong-Toi, Matteo Fontana, Stephan Mandt, Padhraic Smyth, Christoph Lippert
TL;DR
JANET introduces a new conformal prediction framework tailored for time series, providing reliable joint prediction regions with controlled error rates, adaptable to various applications and effective for multi-step ahead forecasting.
Contribution
It generalizes conformal prediction to multivariate time series, enabling efficient, adaptive joint prediction regions with theoretical guarantees for multi-step ahead tasks.
Findings
JANET outperforms existing methods in multi-step prediction accuracy.
It maintains valid error control across diverse datasets.
The framework offers interpretable uncertainty quantification.
Abstract
Conformal prediction provides machine learning models with prediction sets that offer theoretical guarantees, but the underlying assumption of exchangeability limits its applicability to time series data. Furthermore, existing approaches struggle to handle multi-step ahead prediction tasks, where uncertainty estimates across multiple future time points are crucial. We propose JANET (Joint Adaptive predictioN-region Estimation for Time-series), a novel framework for constructing conformal prediction regions that are valid for both univariate and multivariate time series. JANET generalises the inductive conformal framework and efficiently produces joint prediction regions with controlled K-familywise error rates, enabling flexible adaptation to specific application needs. Our empirical evaluation demonstrates JANET's superior performance in multi-step prediction tasks across diverse time…
Peer Reviews
Decision·ICLR 2025 Conference Withdrawn Submission
The paper introduces JANET (Joint Adaptive Prediction-region Estimation for Time-series), a framework designed to construct joint prediction regions (JPRs) for time-series prediction, accommodating both univariate and multivariate data, to multi-step predictions in time series where such assumptions do not hold. JANET offers guarantees by leveraging an inductive conformal prediction (ICP) approach, which only requires a single model fit. Key contributions include adapting non-conformity scores f
However, my concerns are as follows - The theoretical results are a bit limited and incremental compared with existing results; - There are not enough comparisons with state-of-the-art methods in multi-dimensional conformal time-series analysis, such as [42] published in ICML 2024. I believe the method should at least be compared with respect to one-step ahead prediction, since it is a special case of multi-step prediction. There is no fundamental difference in constructing multi-step ahead pre
The authors propose a non-conformity score that effectively controls the *relaxed* familywise error rate for multi-step prediction in an offline setting. This is a clever idea to prevent the excessive conservatism that can result from controlling the exact FWER, which often leads over-coverage.
- It is well-known that split conformal is a special case of full conformal([3], also highlighted directly in [1]). While split conformal offers computational advantages due to requiring only one model fitting, the motivation for framing the discussion solely from the split conformal perspective seems lacking when a more general approach could be described. - The theoretical results are identical to those presented in [1]. A more thorough discussion differentiating this work from [1] would stren
*Motivation*. The paper fills in the gap of generating multi-time-step joint prediction regions in the case of only one time series exists. The authors demonstrated that method can also work well for the case when other independent time series exists for calibration. *Soundness*. The theoretical results in the paper are well argued and sound.
## Clarity questions - Definition 1 / Theorem 1. What is $\mathcal{A}$ and the various constants $\delta$s and $\gamma$s? They are undefined in the paper. What does $\mathcal{A}$ it output, if it only takes two datasets as input and does not take in the test sample? Overall 4.3 was not clear to me. Isn't eq 3 and 4 the same equation with an reused $\epsilon$? Theorem 1 can benefit from some explanation as well (i.e. what does the error depend on, how it behaves with different data properties e
S1. The problem of constructing multi-step prediction regions is important in practice, and uncertainty quantification across multiple sets becomes more challenging due to multiplicity. The concept of K-FWER introduced in this paper is a reasonable approach to effectively quantify error. S2. The paper addresses the conservativeness of existing methods for controlling K-FWER and proposes a direct approach to construct joint prediction regions using a well-designed non-conformity score. Empirical
**Weakness 1.** The paper lacks a concrete theoretical discussion on K-FWER control for the proposed method. The relationship between the designed score and K-FWER control should be made explicit. While I understand the authors treat each time series with horizon H as a single data point and apply conformal inference accordingly, this approach may be unclear to new readers. **Weakness 2.** For a single time series, the K-FWER control remains questionable. The authors should provide thorough the
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsNeural Networks and Applications