Adapting Event Extractors to Medical Data: Bridging the Covariate Shift

TL;DR

This paper explores methods to adapt event extraction models to new medical domains without labeled data by aligning data distributions, introducing new datasets, and evaluating three domain adaptation techniques.

Contribution

It introduces two new medical event extraction datasets and compares three marginal alignment techniques, including a novel instance weighting method, for domain adaptation.

Findings

LIW and DAFT outperform no-transfer baseline

ADA improves only on clinical notes

Best models achieve F1 scores of 70.0 and 72.9

Abstract

We tackle the task of adapting event extractors to new domains without labeled data, by aligning the marginal distributions of source and target domains. As a testbed, we create two new event extraction datasets using English texts from two medical domains: (i) clinical notes, and (ii) doctor-patient conversations. We test the efficacy of three marginal alignment techniques: (i) adversarial domain adaptation (ADA), (ii) domain adaptive fine-tuning (DAFT), and (iii) a novel instance weighting technique based on language model likelihood scores (LIW). LIW and DAFT improve over a no-transfer BERT baseline on both domains, but ADA only improves on clinical notes. Deeper analysis of performance under different types of shifts (e.g., lexical shift, semantic shift) reveals interesting variations among models. Our best-performing models reach F1 scores of 70.0 and 72.9 on notes and conversations respectively, using no labeled data from target domains.