When BERT Fails -- The Limits of EHR Classification

TL;DR

This paper investigates the limitations of BERT-based models in electronic health record classification, identifying specific failure patterns that cause reduced predictive accuracy despite their overall strength.

Contribution

It provides an analysis of failure cases in BERT models for EHR classification, highlighting patterns that lead to performance degradation.

Findings

Identified specific failure patterns in BERT models

Analyzed cases where BERT underperforms in EHR tasks

Provided insights into limitations of transformer models in clinical settings

Abstract

Transformers are powerful text representation learners, useful for all kinds of clinical decision support tasks. Although they outperform baselines on readmission prediction, they are not infallible. Here, we look into one such failure case, and report patterns that lead to inferior predictive performance.