QBERT: Generalist Model for Processing Questions

TL;DR

QBERT is a versatile question-processing model that creates shared representations for various question-related tasks, enabling efficient multi-task learning with performance comparable to specialized models.

Contribution

The paper introduces QBERT, a novel multi-task model that effectively processes questions across diverse tasks, demonstrating the feasibility of a unified question embedding approach.

Findings

QBERT achieves comparable performance to single-task models.

Multi-task training improves efficiency and generalization.

The model effectively handles multiple question-related tasks.

Abstract

Using a single model across various tasks is beneficial for training and applying deep neural sequence models. We address the problem of developing generalist representations of text that can be used to perform a range of different tasks rather than being specialised to a single application. We focus on processing short questions and developing an embedding for these questions that is useful on a diverse set of problems, such as question topic classification, equivalent question recognition, and question answering. This paper introduces QBERT, a generalist model for processing questions. With QBERT, we demonstrate how we can train a multi-task network that performs all question-related tasks and has achieved similar performance compared to its corresponding single-task models.