Robustness and Sensitivity of BERT Models Predicting Alzheimer's Disease from Text

TL;DR

This paper investigates how BERT models used for Alzheimer's disease prediction from text respond to various text alterations, revealing their robustness to linguistic variations but insensitivity to critical clinical information removal.

Contribution

It provides a systematic analysis of BERT's robustness and sensitivity to different types of text modifications in the context of Alzheimer's prediction.

Findings

BERT is robust to natural linguistic variations.

BERT is insensitive to removal of clinically important information.

Abstract

Understanding robustness and sensitivity of BERT models predicting Alzheimer's disease from text is important for both developing better classification models and for understanding their capabilities and limitations. In this paper, we analyze how a controlled amount of desired and undesired text alterations impacts performance of BERT. We show that BERT is robust to natural linguistic variations in text. On the other hand, we show that BERT is not sensitive to removing clinically important information from text.