OSLAT: Open Set Label Attention Transformer for Medical Entity Retrieval and Span Extraction

TL;DR

OSLAT introduces an open set, annotation-free method for medical entity span extraction and linking, leveraging label-attention to learn contextualized representations and implicitly identify spans.

Contribution

The paper presents OSLAT, a novel transformer-based approach that performs open set medical entity linking and span extraction without requiring span annotations.

Findings

OSLAT effectively links entities without span annotations.

It can implicitly learn entity spans during training.

Cross-dataset evaluation shows good generalizability.

Abstract

Medical entity span extraction and linking are critical steps for many healthcare NLP tasks. Most existing entity extraction methods either have a fixed vocabulary of medical entities or require span annotations. In this paper, we propose a method for linking an open set of entities that does not require any span annotations. Our method, Open Set Label Attention Transformer (OSLAT), uses the label-attention mechanism to learn candidate-entity contextualized text representations. We find that OSLAT can not only link entities but is also able to implicitly learn spans associated with entities. We evaluate OSLAT on two tasks: (1) span extraction trained without explicit span annotations, and (2) entity linking trained without span-level annotation. We test the generalizability of our method by training two separate models on two datasets with low entity overlap and comparing cross-dataset performance.