A Framework of Meta Functional Learning for Regularising Knowledge Transfer

TL;DR

This paper introduces Meta Functional Learning (MFL), a novel meta-learning framework that regularizes knowledge transfer to improve classifier performance in data-scarce scenarios like few-shot learning.

Contribution

It proposes a new meta-learning framework for functional regularization, with three variants, enhancing knowledge transfer in few-shot and cross-domain tasks.

Findings

MFL improves classifier performance in few-shot learning.

The variants outperform baseline methods in experiments.

Meta functional regularization enhances knowledge transfer.

Abstract

Machine learning classifiers' capability is largely dependent on the scale of available training data and limited by the model overfitting in data-scarce learning tasks. To address this problem, this work proposes a novel framework of Meta Functional Learning (MFL) by meta-learning a generalisable functional model from data-rich tasks whilst simultaneously regularising knowledge transfer to data-scarce tasks. The MFL computes meta-knowledge on functional regularisation generalisable to different learning tasks by which functional training on limited labelled data promotes more discriminative functions to be learned. Based on this framework, we formulate three variants of MFL: MFL with Prototypes (MFL-P) which learns a functional by auxiliary prototypes, Composite MFL (ComMFL) that transfers knowledge from both functional space and representational space, and MFL with Iterative Updates (MFL-IU) which improves knowledge transfer regularisation from MFL by progressively learning the functional regularisation in knowledge transfer. Moreover, we generalise these variants for knowledge transfer regularisation from binary classifiers to multi-class classifiers. Extensive experiments on two few-shot learning scenarios, Few-Shot Learning (FSL) and Cross-Domain Few-Shot Learning (CD-FSL), show that meta functional learning for knowledge transfer regularisation can improve FSL classifiers.