Team Fusion@ SU@ BC8 SympTEMIST track: transformer-based approach for symptom recognition and linking

TL;DR

This paper introduces a transformer-based method for symptom recognition and linking, combining fine-tuned RoBERTa with BiLSTM-CRF for NER and cosine similarity for entity linking, emphasizing knowledge base importance.

Contribution

It presents a novel combination of transformer fine-tuning and candidate generation for symptom entity recognition and linking in medical texts.

Findings

Knowledge base choice significantly affects accuracy.

Fine-tuning RoBERTa improves NER performance.

Cosine similarity effectively links entities to knowledge bases.

Abstract

This paper presents a transformer-based approach to solving the SympTEMIST named entity recognition (NER) and entity linking (EL) tasks. For NER, we fine-tune a RoBERTa-based (1) token-level classifier with BiLSTM and CRF layers on an augmented train set. Entity linking is performed by generating candidates using the cross-lingual SapBERT XLMR-Large (2), and calculating cosine similarity against a knowledge base. The choice of knowledge base proves to have the highest impact on model accuracy.