Abductive Reasoning as Self-Supervision for Common Sense Question Answering

TL;DR

This paper introduces an abductive reasoning method based on Pattern Theory to enable self-supervised domain adaptation in question answering models, reducing reliance on costly human annotations while maintaining high performance.

Contribution

The paper presents a novel abductive reasoning approach for self-supervised domain adaptation in question answering, leveraging pseudo-labels to match supervised performance levels.

Findings

Self-supervised training retains up to 75% of fully supervised performance.

The approach reduces the need for large annotated datasets.

Experiments on benchmarks validate the effectiveness of the method.

Abstract

Question answering has seen significant advances in recent times, especially with the introduction of increasingly bigger transformer-based models pre-trained on massive amounts of data. While achieving impressive results on many benchmarks, their performances appear to be proportional to the amount of training data available in the target domain. In this work, we explore the ability of current question-answering models to generalize - to both other domains as well as with restricted training data. We find that large amounts of training data are necessary, both for pre-training as well as fine-tuning to a task, for the models to perform well on the designated task. We introduce a novel abductive reasoning approach based on Grenander's Pattern Theory framework to provide self-supervised domain adaptation cues or "pseudo-labels," which can be used instead of expensive human annotations. The proposed self-supervised training regimen allows for effective domain adaptation without losing performance compared to fully supervised baselines. Extensive experiments on two publicly available benchmarks show the efficacy of the proposed approach. We show that neural networks models trained using self-labeled data can retain up to $75\%$ of the performance of models trained on large amounts of human-annotated training data.