Meta-Learning with Heterogeneous Tasks

TL;DR

This paper introduces HeTRoM, a meta-learning method that effectively manages heterogeneous tasks with varying difficulty and noise, improving adaptability and performance in real-world scenarios.

Contribution

The paper proposes a novel rank-based meta-learning approach, HeTRoM, specifically designed to handle heterogeneous tasks and prevent easy tasks from dominating the learning process.

Findings

HeTRoM effectively manages diverse tasks with varying difficulty.

The method improves meta-learner performance in heterogeneous settings.

Experimental results show enhanced adaptability and robustness.

Abstract

Meta-learning is a general approach to equip machine learning models with the ability to handle few-shot scenarios when dealing with many tasks. Most existing meta-learning methods work based on the assumption that all tasks are of equal importance. However, real-world applications often present heterogeneous tasks characterized by varying difficulty levels, noise in training samples, or being distinctively different from most other tasks. In this paper, we introduce a novel meta-learning method designed to effectively manage such heterogeneous tasks by employing rank-based task-level learning objectives, Heterogeneous Tasks Robust Meta-learning (HeTRoM). HeTRoM is proficient in handling heterogeneous tasks, and it prevents easy tasks from overwhelming the meta-learner. The approach allows for an efficient iterative optimization algorithm based on bi-level optimization, which is then improved by integrating statistical guidance. Our experimental results demonstrate that our method provides flexibility, enabling users to adapt to diverse task settings and enhancing the meta-learner's overall performance.