Look to the Right: Mitigating Relative Position Bias in Extractive Question Answering
Kazutoshi Shinoda, Saku Sugawara, Akiko Aizawa

TL;DR
This paper identifies relative position bias as a spurious correlation in extractive QA models and proposes an ensemble-based debiasing method to improve their generalization to unseen relative positions.
Contribution
It introduces a novel bias in QA models related to answer relative positions and presents a debiasing approach that does not require prior distribution knowledge.
Findings
The relative position bias significantly affects model performance on unseen positions.
The proposed ensemble-based method effectively reduces reliance on relative position bias.
Results show improved generalization on biased and full SQuAD datasets.
Abstract
Extractive question answering (QA) models tend to exploit spurious correlations to make predictions when a training set has unintended biases. This tendency results in models not being generalizable to examples where the correlations do not hold. Determining the spurious correlations QA models can exploit is crucial in building generalizable QA models in real-world applications; moreover, a method needs to be developed that prevents these models from learning the spurious correlations even when a training set is biased. In this study, we discovered that the relative position of an answer, which is defined as the relative distance from an answer span to the closest question-context overlap word, can be exploited by QA models as superficial cues for making predictions. Specifically, we find that when the relative positions in a training set are biased, the performance on examples with…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsTopic Modeling · Natural Language Processing Techniques · Speech and dialogue systems
