Medical Spoken Named Entity Recognition

TL;DR

This paper introduces VietMed-NER, the largest Vietnamese spoken medical NER dataset with 18 entity types, and evaluates various pre-trained models, highlighting the superiority of multilingual encoders for speech and text NER tasks.

Contribution

The creation of VietMed-NER, the first large-scale Vietnamese spoken medical NER dataset, and comprehensive baseline evaluations of state-of-the-art models.

Findings

Multilingual models outperform monolingual ones on speech and text NER.

Encoders outperform sequence-to-sequence models in NER tasks.

Translating transcripts enables cross-lingual application of the dataset.

Abstract

Spoken Named Entity Recognition (NER) aims to extract named entities from speech and categorise them into types like person, location, organization, etc. In this work, we present VietMed-NER - the first spoken NER dataset in the medical domain. To our knowledge, our Vietnamese real-world dataset is the largest spoken NER dataset in the world regarding the number of entity types, featuring 18 distinct types. Furthermore, we present baseline results using various state-of-the-art pre-trained models: encoder-only and sequence-to-sequence; and conduct quantitative and qualitative error analysis. We found that pre-trained multilingual models generally outperform monolingual models on reference text and ASR output and encoders outperform sequence-to-sequence models in NER tasks. By translating the transcripts, the dataset can also be utilised for text NER in the medical domain in other languages than Vietnamese. All code, data and models are publicly available: https://github.com/leduckhai/MultiMed/tree/master/VietMed-NER.