Homomorphisms Between Transfer, Multi-Task, and Meta-Learning Systems

TL;DR

This paper formalizes transfer, multi-task, and meta-learning as abstract systems, clarifying their differences and relationships using a systems theory framework to aid interdisciplinary understanding and engineering.

Contribution

It introduces a formal systems-theoretic framework to compare transfer, multi-task, and meta-learning, which was previously underexplored in the literature.

Findings

Formal systems differences are clearly delineated.

Relationships are characterized by composition, hierarchy, and homomorphism.

Input-output system depiction simplifies understanding of concepts.

Abstract

Transfer learning, multi-task learning, and meta-learning are well-studied topics concerned with the generalization of knowledge across learning tasks and are closely related to general intelligence. But, the formal, general systems differences between them are underexplored in the literature. This lack of systems-level formalism leads to difficulties in coordinating related, inter-disciplinary engineering efforts. This manuscript formalizes transfer learning, multi-task learning, and meta-learning as abstract learning systems, consistent with the formal-minimalist abstract systems theory of Mesarovic and Takahara. Moreover, it uses the presented formalism to relate the three concepts of learning in terms of composition, hierarchy, and structural homomorphism. Findings are readily depicted in terms of input-output systems, highlighting the ease of delineating formal, general systems differences between transfer, multi-task, and meta-learning.