Biomedical NER for the Enterprise with Distillated BERN2 and the Kazu Framework

TL;DR

This paper introduces Kazu, an extensible and scalable open-source framework for biomedical named entity recognition tailored for the pharmaceutical industry, built around a computationally efficient version of BERN2.

Contribution

The paper presents Kazu, a novel framework integrating TinyBERN2 and multiple BioNLP tools, addressing industry-specific needs unmet by existing open-source systems.

Findings

Kazu effectively supports pharmaceutical BioNLP tasks.

TinyBERN2 improves NER efficiency and scalability.

Kazu integrates multiple BioNLP tools into a unified system.

Abstract

In order to assist the drug discovery/development process, pharmaceutical companies often apply biomedical NER and linking techniques over internal and public corpora. Decades of study of the field of BioNLP has produced a plethora of algorithms, systems and datasets. However, our experience has been that no single open source system meets all the requirements of a modern pharmaceutical company. In this work, we describe these requirements according to our experience of the industry, and present Kazu, a highly extensible, scalable open source framework designed to support BioNLP for the pharmaceutical sector. Kazu is a built around a computationally efficient version of the BERN2 NER model (TinyBERN2), and subsequently wraps several other BioNLP technologies into one coherent system. KAZU framework is open-sourced: https://github.com/AstraZeneca/KAZU