Chameleon: Learning Model Initializations Across Tasks With Different Schemas

TL;DR

Chameleon introduces a meta-learning model that learns to initialize neural networks across tasks with varying schemas, enabling effective few-shot learning on diverse datasets with different predictor structures.

Contribution

It is the first method to learn model initializations across tasks with different schemas, addressing a key limitation in existing meta-learning approaches.

Findings

Successfully learned initializations across 23 datasets.

Enabled few-shot classification with unstructured data.

Demonstrated effectiveness on diverse schemas.

Abstract

Parametric models, and particularly neural networks, require weight initialization as a starting point for gradient-based optimization. Recent work shows that a specific initial parameter set can be learned from a population of supervised learning tasks. Using this initial parameter set enables a fast convergence for unseen classes even when only a handful of instances is available (model-agnostic meta-learning). Currently, methods for learning model initializations are limited to a population of tasks sharing the same schema, i.e., the same number, order, type, and semantics of predictor and target variables. In this paper, we address the problem of meta-learning parameter initialization across tasks with different schemas, i.e., if the number of predictors varies across tasks, while they still share some variables. We propose Chameleon, a model that learns to align different predictor schemas to a common representation. In experiments on 23 datasets of the OpenML-CC18 benchmark, we show that Chameleon can successfully learn parameter initializations across tasks with different schemas, presenting, to the best of our knowledge, the first cross-dataset few-shot classification approach for unstructured data.