Model-Agnostic Meta-Learning for Natural Language Understanding Tasks in Finance

TL;DR

This paper applies model-agnostic meta-learning (MAML) to low-resource financial natural language understanding tasks, demonstrating improved adaptation and state-of-the-art results across diverse datasets and real-world scenarios.

Contribution

It investigates MAML's effectiveness in financial NLU, covering multiple task types and a real stock prediction scenario, showing enhanced adaptation in low-resource settings.

Findings

MAML achieves state-of-the-art performance on financial NLU datasets.

MAML adapts quickly to low-resource tasks and diverse datasets.

The approach outperforms traditional fine-tuning methods.

Abstract

Natural language understanding(NLU) is challenging for finance due to the lack of annotated data and the specialized language in that domain. As a result, researchers have proposed to use pre-trained language model and multi-task learning to learn robust representations. However, aggressive fine-tuning often causes over-fitting and multi-task learning may favor tasks with significantly larger amounts data, etc. To address these problems, in this paper, we investigate model-agnostic meta-learning algorithm(MAML) in low-resource financial NLU tasks. Our contribution includes: 1. we explore the performance of MAML method with multiple types of tasks: GLUE datasets, SNLI, Sci-Tail and Financial PhraseBank; 2. we study the performance of MAML method with multiple single-type tasks: a real scenario stock price prediction problem with twitter text data. Our models achieve the state-of-the-art performance according to the experimental results, which demonstrate that our method can adapt fast and well to low-resource situations.