BERTops: Studying BERT Representations under a Topological Lens

TL;DR

This paper introduces a topological analysis method for BERT's hidden representations using persistent homology, providing a new scoring function that correlates with model accuracy and robustness against adversarial attacks.

Contribution

It proposes a novel persistence scoring function (PSF) that captures topological features of BERT representations, outperforming existing metrics in accuracy correlation and robustness prediction.

Findings

PSF accurately correlates with test set accuracy.

PSF is more stable to perturbations than baseline metrics.

PSF predicts attack success rates effectively.

Abstract

Proposing scoring functions to effectively understand, analyze and learn various properties of high dimensional hidden representations of large-scale transformer models like BERT can be a challenging task. In this work, we explore a new direction by studying the topological features of BERT hidden representations using persistent homology (PH). We propose a novel scoring function named "persistence scoring function (PSF)" which: (i) accurately captures the homology of the high-dimensional hidden representations and correlates well with the test set accuracy of a wide range of datasets and outperforms existing scoring metrics, (ii) captures interesting post fine-tuning "per-class" level properties from both qualitative and quantitative viewpoints, (iii) is more stable to perturbations as compared to the baseline functions, which makes it a very robust proxy, and (iv) finally, also serves as a predictor of the attack success rates for a wide category of black-box and white-box adversarial attack methods. Our extensive correlation experiments demonstrate the practical utility of PSF on various NLP tasks relevant to BERT.