Term Expansion and FinBERT fine-tuning for Hypernym and Synonym Ranking of Financial Terms

TL;DR

This paper presents a system for hypernym and synonym ranking of financial terms using fine-tuned transformer models and data augmentation, achieving top performance in the FinSim-3 shared task.

Contribution

It introduces a novel approach combining term expansion, data augmentation, and transformer fine-tuning for improved financial term semantic matching.

Findings

Data augmentation improves semantic similarity tasks.

Fine-tuning SentenceBERT yields high accuracy (0.917).

Combining FinBERT with data sources enhances performance.

Abstract

Hypernym and synonym matching are one of the mainstream Natural Language Processing (NLP) tasks. In this paper, we present systems that attempt to solve this problem. We designed these systems to participate in the FinSim-3, a shared task of FinNLP workshop at IJCAI-2021. The shared task is focused on solving this problem for the financial domain. We experimented with various transformer based pre-trained embeddings by fine-tuning these for either classification or phrase similarity tasks. We also augmented the provided dataset with abbreviations derived from prospectus provided by the organizers and definitions of the financial terms from DBpedia [Auer et al., 2007], Investopedia, and the Financial Industry Business Ontology (FIBO). Our best performing system uses both FinBERT [Araci, 2019] and data augmentation from the afore-mentioned sources. We observed that term expansion using data augmentation in conjunction with semantic similarity is beneficial for this task and could be useful for the other tasks that deal with short phrases. Our best performing model (Accuracy: 0.917, Rank: 1.156) was developed by fine-tuning SentenceBERT [Reimers et al., 2019] (with FinBERT at the backend) over an extended labelled set created using the hierarchy of labels present in FIBO.