Evaluating Data Influence in Meta Learning
Chenyang Ren, Huanyi Xie, Shu Yang, Meng Ding, Lijie Hu, Di Wang
TL;DR
This paper introduces a novel influence-based data attribution framework for meta learning, accurately assessing data contributions in bilevel optimization, improving training efficiency and robustness.
Contribution
It proposes a general influence function-based framework with task and instance influence functions for meta learning, addressing dual-layer complexity.
Findings
Effective data influence evaluation demonstrated in downstream tasks
Framework captures both direct and indirect data effects
Enhances training efficiency and robustness in meta learning
Abstract
As one of the most fundamental models, meta learning aims to effectively address few-shot learning challenges. However, it still faces significant issues related to the training data, such as training inefficiencies due to numerous low-contribution tasks in large datasets and substantial noise from incorrect labels. Thus, training data attribution methods are needed for meta learning. However, the dual-layer structure of mata learning complicates the modeling of training data contributions because of the interdependent influence between meta-parameters and task-specific parameters, making existing data influence evaluation tools inapplicable or inaccurate. To address these challenges, based on the influence function, we propose a general data attribution evaluation framework for meta-learning within the bilevel optimization framework. Our approach introduces task influence functions…
Peer Reviews
Decision·ICLR 2026 Poster
- The paper cleanly formulates meta-learning as bilevel optimization and introduces complementary task- and instance-level influence measures tailored to that structure. - It contributes practical scalability via acceleration techniques, making the closed-form influence derivations computationally feasible. - The experimental setup is comprehensive.
- The paper lacks a true ablation study isolating the contribution of its key components. - Accuracy gains over strong baselines are modest and sometimes below retraining, with the main advantage being runtime. - Reproducibility is constrained by the absence of a code repository or data link in the submission.
1. Introduced influence functions into meta-learning and adapted them to the bilevel optimization structure. 2. Clarified why directly applying standard influence functions in bilevel optimization is incorrect and derived the corrected Task-IF formulation that properly accounts for inner–outer dependencies. 3. Designed two types of influence functions, Task-IF and Instance-IF, enabling multi-granularity data evaluation.
1. The proposed method for training instance influence, which models the outer-loss change as a simple additive proxy P, implicitly relies on the sufficiency of local, linear approximations. This two-stage linear approximation (first to compute P, then to compute its influence) may fail to capture the complex, non-linear effects of removing a data point, especially in optimization landscapes with high curvature or when the removal of a point induces a significant shift in the task-specific param
1. Novelty: The paper is the first to formally and introduce influence functions into the bilevel optimization framework for meta-learning, to thoroughly include task influence and data(training/validation) influence. 2. Comprehensive and Well-Structured Framework: The proposed framework is comprehensive, addressing data influence at both the task and instance levels, and for both training and validation data. The distinction between these different levels of influence is well-motivated and cl
1. The scope: The proposed IF evaluation method is only applicable for second-order MAML, which is only one specific algorithm among numerous meta-learning algorithms. A narrowed title and scope, e.g. "Evaluating data influence in MAML by IF" would be more proper and accurate. 2. Experimental and practical concern: The burden calculating IF is concerned, though some approximation and acceleration techniques have been applied. It seems there are much more simple and intuitive baselines in the a
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsMachine Learning and Data Classification · Online Learning and Analytics