ConveRT for FAQ Answering

TL;DR

This paper introduces a new pre-training method to adapt the ConveRT conversational retrieval model for multilingual FAQ answering, demonstrated on Dutch COVID-19 vaccine FAQs, outperforming existing open-source models.

Contribution

A novel pre-training approach for adapting ConveRT to low-resource languages, applied to Dutch FAQ answering about COVID-19 vaccines.

Findings

Outperforms open-source alternatives in low-data settings

Effective in high-data regimes as well

First application of ConveRT adaptation to Dutch FAQ task

Abstract

Knowledgeable FAQ chatbots are a valuable resource to any organization. While powerful and efficient retrieval-based models exist for English, it is rarely the case for other languages for which the same amount of training data is not available. In this paper, we propose a novel pre-training procedure to adapt ConveRT, an English conversational retriever model, to other languages with less training data available. We apply it for the first time to the task of Dutch FAQ answering related to the COVID-19 vaccine. We show it performs better than an open-source alternative in both a low-data regime and a high-data regime.