KnowNER: Incremental Multilingual Knowledge in Named Entity Recognition

TL;DR

KnowNER is a multilingual NER system that systematically integrates various external knowledge sources through a modular framework, significantly improving accuracy across multiple languages.

Contribution

It introduces a novel modular framework for incorporating different depths of external knowledge into multilingual NER using CRFs, enhancing performance.

Findings

Systematic boost in NER accuracy with deeper knowledge integration

Competitive performance across English, German, and Spanish

Flexible framework adaptable to multiple languages

Abstract

KnowNER is a multilingual Named Entity Recognition (NER) system that leverages different degrees of external knowledge. A novel modular framework divides the knowledge into four categories according to the depth of knowledge they convey. Each category consists of a set of features automatically generated from different information sources (such as a knowledge-base, a list of names or document-specific semantic annotations) and is used to train a conditional random field (CRF). Since those information sources are usually multilingual, KnowNER can be easily trained for a wide range of languages. In this paper, we show that the incorporation of deeper knowledge systematically boosts accuracy and compare KnowNER with state-of-the-art NER approaches across three languages (i.e., English, German and Spanish) performing amongst state-of-the art systems in all of them.