On the Importance of Delexicalization for Fact Verification
Sandeep Suntwal, Mithun Paul, Rebecca Sharp, Mihai Surdeanu

TL;DR
This paper investigates the reliance of fact verification models on lexical cues, especially POS tags, and proposes masking strategies to improve domain transferability and reduce lexical dependence.
Contribution
It analyzes attention weights in RTE models for fact verification and introduces masking techniques to mitigate lexical dependence, enhancing cross-domain performance.
Findings
Attention weights focus on nouns and POS tags.
Lexicalized models transfer poorly across domains.
Masking names improves cross-domain accuracy.
Abstract
In this work we aim to understand and estimate the importance that a neural network assigns to various aspects of the data while learning and making predictions. Here we focus on the recognizing textual entailment (RTE) task and its application to fact verification. In this context, the contributions of this work are as follows. We investigate the attention weights a state of the art RTE method assigns to input tokens in the RTE component of fact verification systems, and confirm that most of the weight is assigned to POS tags of nouns (e.g., NN, NNP etc.) or their phrases. To verify that these lexicalized models transfer poorly, we implement a domain transfer experiment where a RTE component is trained on the FEVER data, and tested on the Fake News Challenge (FNC) dataset. As expected, even though this method achieves high accuracy when evaluated in the same domain, the performance in…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
