Cascaded Transfer: Learning Many Tasks under Budget Constraints

TL;DR

This paper introduces Cascaded Transfer Learning, a hierarchical approach for multi-task learning that efficiently allocates training resources across tasks organized in a tree structure, improving accuracy and cost-effectiveness.

Contribution

The paper proposes a novel cascaded transfer paradigm with a tree-based structure for multi-task learning under budget constraints, enabling hierarchical information transfer.

Findings

Improved accuracy over baseline methods

More cost-effective training process

Effective transfer across large task collections

Abstract

Many-Task Learning refers to the setting where a large number of related tasks need to be learned, the exact relationships between tasks are not known. We introduce the Cascaded Transfer Learning, a novel many-task transfer learning paradigm where information (e.g. model parameters) cascades hierarchically through tasks that are learned by individual models of the same class, while respecting given budget constraints. The cascade is organized as a rooted tree that specifies the order in which tasks are learned and refined. We design a cascaded transfer mechanism deployed over a minimum spanning tree structure that connects the tasks according to a suitable distance measure, and allocates the available training budget along its branches. Experiments on synthetic and real many-task settings show that the resulting method enables more accurate and cost effective adaptation across large task collections compared to alternative approaches.