"I've Heard of You!": Generate Spoken Named Entity Recognition Data for Unseen Entities

TL;DR

This paper presents a cost-effective method for generating spoken NER data for unseen entities using large language models and TTS, improving performance in zero-shot and domain adaptation scenarios.

Contribution

It introduces a novel data generation approach leveraging LLMs and TTS for spoken NER, along with a noise filtering metric, and releases a new benchmark dataset.

Findings

Achieves state-of-the-art results in multiple settings

Effective in zero-shot domain adaptation

Reduces annotation costs significantly

Abstract

Spoken named entity recognition (NER) aims to identify named entities from speech, playing an important role in speech processing. New named entities appear every day, however, annotating their Spoken NER data is costly. In this paper, we demonstrate that existing Spoken NER systems perform poorly when dealing with previously unseen named entities. To tackle this challenge, we propose a method for generating Spoken NER data based on a named entity dictionary (NED) to reduce costs. Specifically, we first use a large language model (LLM) to generate sentences from the sampled named entities and then use a text-to-speech (TTS) system to generate the speech. Furthermore, we introduce a noise metric to filter out noisy data. To evaluate our approach, we release a novel Spoken NER benchmark along with a corresponding NED containing 8,853 entities. Experiment results show that our method achieves state-of-the-art (SOTA) performance in the in-domain, zero-shot domain adaptation, and fully zero-shot settings. Our data will be available at https://github.com/DeepLearnXMU/HeardU.