Focusing on Potential Named Entities During Active Label Acquisition

TL;DR

This paper introduces novel active learning query functions for NER that focus on potential positive tokens, reducing annotation effort while maintaining or improving model performance across multiple datasets.

Contribution

It proposes new AL sentence query evaluation functions emphasizing potential named entities and a normalization method to improve data efficiency in NER tasks.

Findings

Reduces annotated tokens needed for effective NER

Achieves better or comparable performance with fewer labels

Effective across multiple domain datasets

Abstract

Named entity recognition (NER) aims to identify mentions of named entities in an unstructured text and classify them into predefined named entity classes. While deep learning-based pre-trained language models help to achieve good predictive performances in NER, many domain-specific NER applications still call for a substantial amount of labeled data. Active learning (AL), a general framework for the label acquisition problem, has been used for NER tasks to minimize the annotation cost without sacrificing model performance. However, the heavily imbalanced class distribution of tokens introduces challenges in designing effective AL querying methods for NER. We propose several AL sentence query evaluation functions that pay more attention to potential positive tokens, and evaluate these proposed functions with both sentence-based and token-based cost evaluation strategies. We also propose a better data-driven normalization approach to penalize sentences that are too long or too short. Our experiments on three datasets from different domains reveal that the proposed approach reduces the number of annotated tokens while achieving better or comparable prediction performance with conventional methods.