FAQ Retrieval using Query-Question Similarity and BERT-Based Query-Answer Relevance

TL;DR

This paper presents a FAQ retrieval system combining unsupervised query-question similarity with BERT-based query-answer relevance, effectively handling limited labeled data and outperforming baselines on Japanese and English datasets.

Contribution

The paper introduces a novel FAQ retrieval approach that leverages unsupervised similarity measures and BERT for relevance, addressing data scarcity issues.

Findings

Outperforms baseline methods on localgovFAQ and StackExchange datasets.

Effectively combines unsupervised similarity with BERT-based relevance.

Demonstrates applicability in multilingual and domain-specific contexts.

Abstract

Frequently Asked Question (FAQ) retrieval is an important task where the objective is to retrieve an appropriate Question-Answer (QA) pair from a database based on a user's query. We propose a FAQ retrieval system that considers the similarity between a user's query and a question as well as the relevance between the query and an answer. Although a common approach to FAQ retrieval is to construct labeled data for training, it takes annotation costs. Therefore, we use a traditional unsupervised information retrieval system to calculate the similarity between the query and question. On the other hand, the relevance between the query and answer can be learned by using QA pairs in a FAQ database. The recently-proposed BERT model is used for the relevance calculation. Since the number of QA pairs in FAQ page is not enough to train a model, we cope with this issue by leveraging FAQ sets that are similar to the one in question. We evaluate our approach on two datasets. The first one is localgovFAQ, a dataset we construct in a Japanese administrative municipality domain. The second is StackExchange dataset, which is the public dataset in English. We demonstrate that our proposed method outperforms baseline methods on these datasets.