Meta-RTL: Reinforcement-Based Meta-Transfer Learning for Low-Resource Commonsense Reasoning

TL;DR

Meta-RTL introduces a reinforcement learning framework that dynamically weights source tasks in meta-transfer learning, significantly improving low-resource commonsense reasoning performance using BERT and ALBERT.

Contribution

It proposes a novel reinforcement-based approach to estimate source task relevance, enhancing knowledge transfer in low-resource settings.

Findings

Meta-RTL outperforms existing methods on benchmark datasets.

The approach achieves larger gains in extremely low-resource scenarios.

Reinforcement-based task weighting improves transfer learning effectiveness.

Abstract

Meta learning has been widely used to exploit rich-resource source tasks to improve the performance of low-resource target tasks. Unfortunately, most existing meta learning approaches treat different source tasks equally, ignoring the relatedness of source tasks to the target task in knowledge transfer. To mitigate this issue, we propose a reinforcement-based multi-source meta-transfer learning framework (Meta-RTL) for low-resource commonsense reasoning. In this framework, we present a reinforcement-based approach to dynamically estimating source task weights that measure the contribution of the corresponding tasks to the target task in the meta-transfer learning. The differences between the general loss of the meta model and task-specific losses of source-specific temporal meta models on sampled target data are fed into the policy network of the reinforcement learning module as rewards. The policy network is built upon LSTMs that capture long-term dependencies on source task weight estimation across meta learning iterations. We evaluate the proposed Meta-RTL using both BERT and ALBERT as the backbone of the meta model on three commonsense reasoning benchmark datasets. Experimental results demonstrate that Meta-RTL substantially outperforms strong baselines and previous task selection strategies and achieves larger improvements on extremely low-resource settings.