'Finance Wizard' at the FinLLM Challenge Task: Financial Text Summarization

TL;DR

This paper describes a pipeline for financial text summarization using a fine-tuned Llama3 8B model, achieving competitive results in the FinNLP-AgentScen 2024 challenge.

Contribution

It introduces a multi-stage fine-tuning process for adapting foundation models to financial summarization tasks, including continued pre-training and instruction-tuning.

Findings

Secured third place in the challenge with a ROUGE-1 score of 0.521.

Demonstrated effectiveness of multi-task instruction-tuning for financial NLP.

Achieved competitive performance with a specialized, task-specific model.

Abstract

This paper presents our participation under the team name `Finance Wizard' in the FinNLP-AgentScen 2024 shared task #2: Financial Text Summarization. It documents our pipeline approach of fine-tuning a foundation model into a task-specific model for Financial Text Summarization. It involves (1) adapting Llama3 8B, a foundation model, to the Finance domain via continued pre-training, (2) multi-task instruction-tuning to further equip the model with more finance-related capabilities, (3) finally fine-tuning the model into a task-specific `expert'. Our model, FinLlama3\_sum, yielded commendable results, securing the third position in its category with a ROUGE-1 score of 0.521.