Can humans help BERT gain "confidence"?

TL;DR

This paper explores integrating cognitive features like EEG and eye-tracking data into BERT to enhance its performance, improve robustness, and develop explainability methods, bridging AI and neuroscience.

Contribution

It introduces a novel approach of combining cognitive data with BERT, demonstrating performance improvements and proposing a new explainability framework based on cognitive features.

Findings

Cognitive features from EEG and eye-tracking improve BERT's NLP performance.

A robustness pipeline confirms the performance gains.

A word-EEG lexicon is derived for external dataset benchmarking.

Abstract

The advancements in artificial intelligence over the last decade have opened a multitude of avenues for interdisciplinary research. Since the idea of artificial intelligence was inspired by the working of neurons in the brain, it seems pretty practical to combine the two fields and take the help of cognitive data to train AI models. Not only it will help to get a deeper understanding of the technology, but of the brain as well. In this thesis, I conduct novel experiments to integrate cognitive features from the Zurich Cognitive Corpus (ZuCo) (Hollenstein et al., 2018) with a transformer-based encoder model called BERT. I show how EEG and eye-tracking features from ZuCo can help to increase the performance of the NLP model. I confirm the performance increase with the help of a robustness-checking pipeline and derive a word-EEG lexicon to use in benchmarking on an external dataset that does not have any cognitive features associated with it. Further, I analyze the internal working mechanism of BERT and explore a potential method for model explainability by correlating it with a popular model-agnostic explainability framework called LIME (Ribeiro et al., 2016). Finally, I discuss the possible directions to take this research forward.