DSC IIT-ISM at SemEval-2020 Task 6: Boosting BERT with Dependencies for Definition Extraction

TL;DR

This paper enhances BERT for definition extraction by integrating dependency structures via a joint BERT and Graph Convolutional Network model, improving performance on SemEval 2020 tasks.

Contribution

It introduces a novel joint model combining BERT with dependency-aware graph convolutional networks for better definition extraction.

Findings

The joint model outperforms standalone BERT in definition extraction tasks.

Achieves results comparable to fine-tuned BERT models on SemEval 2020.

Demonstrates the effectiveness of incorporating dependency information into transformer models.

Abstract

We explore the performance of Bidirectional Encoder Representations from Transformers (BERT) at definition extraction. We further propose a joint model of BERT and Text Level Graph Convolutional Network so as to incorporate dependencies into the model. Our proposed model produces better results than BERT and achieves comparable results to BERT with fine tuned language model in DeftEval (Task 6 of SemEval 2020), a shared task of classifying whether a sentence contains a definition or not (Subtask 1).