Sample Efficient Subspace-based Representations for Nonlinear Meta-Learning

TL;DR

This paper develops a framework for learning sample-efficient, subspace-based representations for nonlinear meta-learning tasks, extending previous linear approaches to more complex models like neural networks and classification.

Contribution

It introduces a method to learn subspace representations for nonlinear tasks, providing theoretical guarantees and empirical validation, which was lacking in prior linear-focused work.

Findings

Subspace representations can be learned efficiently for nonlinear tasks.

The approach improves sample complexity for future tasks.

Empirical results confirm theoretical predictions in classification and neural networks.

Abstract

Constructing good representations is critical for learning complex tasks in a sample efficient manner. In the context of meta-learning, representations can be constructed from common patterns of previously seen tasks so that a future task can be learned quickly. While recent works show the benefit of subspace-based representations, such results are limited to linear-regression tasks. This work explores a more general class of nonlinear tasks with applications ranging from binary classification, generalized linear models and neural nets. We prove that subspace-based representations can be learned in a sample-efficient manner and provably benefit future tasks in terms of sample complexity. Numerical results verify the theoretical predictions in classification and neural-network regression tasks.