MINER: Improving Out-of-Vocabulary Named Entity Recognition from an Information Theoretic Perspective

TL;DR

MINER is a new NER framework that improves out-of-vocabulary entity recognition by using mutual information objectives to enhance context understanding and reduce reliance on memorized entity names.

Contribution

It introduces an information-theoretic approach with mutual information objectives to improve OOV NER performance, addressing over-reliance on entity mention memorization.

Findings

Better OOV entity recognition performance across datasets

Effective use of mutual information for NER training

Reduces bias from memorized entity names

Abstract

NER model has achieved promising performance on standard NER benchmarks. However, recent studies show that previous approaches may over-rely on entity mention information, resulting in poor performance on out-of-vocabulary (OOV) entity recognition. In this work, we propose MINER, a novel NER learning framework, to remedy this issue from an information-theoretic perspective. The proposed approach contains two mutual information-based training objectives: i) generalizing information maximization, which enhances representation via deep understanding of context and entity surface forms; ii) superfluous information minimization, which discourages representation from rote memorizing entity names or exploiting biased cues in data. Experiments on various settings and datasets demonstrate that it achieves better performance in predicting OOV entities.