Zero-Shot Estimation of Base Models' Weights in Ensemble of Machine Reading Comprehension Systems for Robust Generalization

TL;DR

This paper introduces a zero-shot weighted ensemble approach to enhance out-of-domain robustness of machine reading comprehension models, effectively improving accuracy and domain adaptability without additional training.

Contribution

It proposes a novel zero-shot weight estimation method for ensemble models, addressing out-of-domain generalization in MRC systems.

Findings

Improved accuracy on out-of-domain datasets

Enhanced robustness against domain shifts

Effective ensemble aggregation based on estimated weights

Abstract

One of the main challenges of the machine reading comprehension (MRC) models is their fragile out-of-domain generalization, which makes these models not properly applicable to real-world general-purpose question answering problems. In this paper, we leverage a zero-shot weighted ensemble method for improving the robustness of out-of-domain generalization in MRC models. In the proposed method, a weight estimation module is used to estimate out-of-domain weights, and an ensemble module aggregate several base models' predictions based on their weights. The experiments indicate that the proposed method not only improves the final accuracy, but also is robust against domain changes.