An Automated Question-Answering Framework Based on Evolution Algorithm

TL;DR

This paper introduces an automated question-answering framework that uses an evolution algorithm to adapt network architectures across multiple datasets, reducing human effort and improving performance.

Contribution

It presents a novel evolution algorithm incorporating prior knowledge and performance estimation to automatically optimize QA model architectures for various datasets.

Findings

Achieves 78.9 EM and 86.1 F1 on SQuAD 1.1

Attains 69.9 EM and 72.5 F1 on SQuAD 2.0

Reaches 47.0 EM and 62.9 F1 on NewsQA

Abstract

Building a deep learning model for a Question-Answering (QA) task requires a lot of human effort, it may need several months to carefully tune various model architectures and find a best one. It's even harder to find different excellent models for multiple datasets. Recent works show that the best model structure is related to the dataset used, and one single model cannot adapt to all tasks. In this paper, we propose an automated Question-Answering framework, which could automatically adjust network architecture for multiple datasets. Our framework is based on an innovative evolution algorithm, which is stable and suitable for multiple dataset scenario. The evolution algorithm for search combine prior knowledge into initial population and use a performance estimator to avoid inefficient mutation by predicting the performance of candidate model architecture. The prior knowledge used in initial population could improve the final result of the evolution algorithm. The performance estimator could quickly filter out models with bad performance in population as the number of trials increases, to speed up the convergence. Our framework achieves 78.9 EM and 86.1 F1 on SQuAD 1.1, 69.9 EM and 72.5 F1 on SQuAD 2.0. On NewsQA dataset, the found model achieves 47.0 EM and 62.9 F1.