Self-Supervised Test-Time Learning for Reading Comprehension

TL;DR

This paper introduces a self-supervised test-time learning approach for reading comprehension that adapts to individual contexts without large-scale labeled datasets, achieving competitive accuracy with supervised methods.

Contribution

The work presents a novel test-time learning method that trains models on synthetic question-answer pairs for each context, eliminating the need for extensive labeled datasets.

Findings

Achieves accuracy comparable to supervised methods.

Outperforms existing unsupervised reading comprehension techniques.

Smaller models also perform competitively.

Abstract

Recent work on unsupervised question answering has shown that models can be trained with procedurally generated question-answer pairs and can achieve performance competitive with supervised methods. In this work, we consider the task of unsupervised reading comprehension and present a method that performs "test-time learning" (TTL) on a given context (text passage), without requiring training on large-scale human-authored datasets containing \textit{context-question-answer} triplets. This method operates directly on a single test context, uses self-supervision to train models on synthetically generated question-answer pairs, and then infers answers to unseen human-authored questions for this context. Our method achieves accuracies competitive with fully supervised methods and significantly outperforms current unsupervised methods. TTL methods with a smaller model are also competitive with the current state-of-the-art in unsupervised reading comprehension.