An Ensemble Approach to Acronym Extraction using Transformers

TL;DR

This paper presents an ensemble method combining transformer-based and neural network approaches for automatic acronym extraction, improving performance across multiple languages and domains.

Contribution

It introduces a novel ensemble approach utilizing transformers and CNNs, along with augmented datasets, for more accurate acronym extraction.

Findings

Achieved macro-F1 scores between 0.57 and 0.74 across languages.

Enhanced dataset with additional samples improved extraction performance.

Publicly released code and models for reproducibility.

Abstract

Acronyms are abbreviated units of a phrase constructed by using initial components of the phrase in a text. Automatic extraction of acronyms from a text can help various Natural Language Processing tasks like machine translation, information retrieval, and text summarisation. This paper discusses an ensemble approach for the task of Acronym Extraction, which utilises two different methods to extract acronyms and their corresponding long forms. The first method utilises a multilingual contextual language model and fine-tunes the model to perform the task. The second method relies on a convolutional neural network architecture to extract acronyms and append them to the output of the previous method. We also augment the official training dataset with additional training samples extracted from several open-access journals to help improve the task performance. Our dataset analysis also highlights the noise within the current task dataset. Our approach achieves the following macro-F1 scores on test data released with the task: Danish (0.74), English-Legal (0.72), English-Scientific (0.73), French (0.63), Persian (0.57), Spanish (0.65), Vietnamese (0.65). We release our code and models publicly.