TL;DR
This paper introduces Dimension Agnostic Neural Processes (DANP), a novel meta-learning model that handles diverse input dimensions and learns generalizable features, outperforming previous neural process methods in regression tasks.
Contribution
DANP incorporates a Dimension Aggregator Block and Transformer architecture to improve neural processes' flexibility and applicability across diverse regression datasets.
Findings
DANP outperforms previous NP models on synthetic and real regression tasks.
The model effectively handles diverse input dimensions.
DANP demonstrates broad applicability in various regression scenarios.
Abstract
Meta-learning aims to train models that can generalize to new tasks with limited labeled data by extracting shared features across diverse task datasets. Additionally, it accounts for prediction uncertainty during both training and evaluation, a concept known as uncertainty-aware meta-learning. Neural Process(NP) is a well-known uncertainty-aware meta-learning method that constructs implicit stochastic processes using parametric neural networks, enabling rapid adaptation to new tasks. However, existing NP methods face challenges in accommodating diverse input dimensions and learned features, limiting their broad applicability across regression tasks. To address these limitations and advance the utility of NP models as general regressors, we introduce Dimension Agnostic Neural Processes(DANP). DANP incorporates Dimension Aggregator Block(DAB) to transform input features into a…
Peer Reviews
Decision·ICLR 2025 Poster
Originality - The paper seem it has an evident level of novelty, tackling the diverse input and output dimensions challenge in the uncertainty aware meta-learning methods such as neural processes. Two novelties seems to be the case here, the dimension aggregation block and the latent path, in a transformer-like arhitecture. Quality - The paper is well motivated, structured and presented, the problem is well introduces and connected to existing work. The writing is good. There is extensive and
Presentation of the tasks/problems that the method addresses - The presentation is sufficiently clear. I find that more on the actual task considered here can help to appreciate more the significance and the benefits of this approach. In particular related how the GP regression and the image completion tasks benchmarks help to validate the broad applicability of the approach? Evaluations - Non consistent result and seem to be marginal improvements in the GP Regression (from-scratch case) and t
1. Introduces the DAB module, enabling the model to handle inputs and outputs of varying dimensions, adding flexibility. 2. Covers multiple tasks and scenarios, demonstrating the model's stability across different conditions. 3. Performs well in regression, hyperparameter tuning, and other tasks, showing promise for broad applications.
1. Model Complexity. The design is complex, making replication and understanding challenging. 2. Lacks Analysis of Computational Costs. There’s no discussion of the model's time and resource requirements, impacting assessments for practical use. 3. Limited Application Scope. Primarily validated on regression tasks, with little exploration of classification or other tasks.
- DANP is a novel extension of NP, that addresses the limitations of existing NP methods in handling diverse input dimensions and learned features. - This work not only points out the shortcomings of current NP methods but also proposes a robust solution through the DAB and the integration of Transformer architecture. - The paper is clear in its structure and presentation.
- The paper focuses on regression tasks, but its applicability to other tasks such as classification is not thoroughly explored. It could benefit from additional experiments or a theoretical discussion on how DANP might perform in non-regression tasks. - While DANP shows promising results, the paper lacks a detailed discussion on the model's interpretability. The paper should include an analysis or discussion on how the components of DANP contribute to its predictions, especially given its compl
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
