MetaXT: Meta Cross-Task Transfer between Disparate Label Spaces
Srinagesh Sharma, Guoqing Zheng, Ahmed Hassan Awadallah

TL;DR
MetaXT introduces a novel bi-level optimization framework that enables effective transfer learning across NLP tasks with different label spaces, significantly improving performance in few-shot scenarios.
Contribution
The paper proposes MetaXT, a method that transfers knowledge between tasks with disparate label spaces using a label transfer network and bi-level optimization.
Findings
MetaXT outperforms baseline methods in cross-task transfer settings.
Effective in low-resource, few-shot learning scenarios.
Demonstrates robustness across four NLP tasks with label space disparities.
Abstract
Albeit the universal representational power of pre-trained language models, adapting them onto a specific NLP task still requires a considerably large amount of labeled data. Effective task fine-tuning meets challenges when only a few labeled examples are present for the task. In this paper, we aim to the address of the problem of few shot task learning by exploiting and transferring from a different task which admits a related but disparate label space. Specifically, we devise a label transfer network (LTN) to transform the labels from source task to the target task of interest for training. Both the LTN and the model for task prediction are learned via a bi-level optimization framework, which we term as MetaXT. MetaXT offers a principled solution to best adapt a pre-trained language model to the target task by transferring knowledge from the source task. Empirical evaluations on…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsDomain Adaptation and Few-Shot Learning · Multimodal Machine Learning Applications · Topic Modeling
