Generative Pre-trained Transformer for Vietnamese Community-based COVID-19 Question Answering

TL;DR

This paper demonstrates that GPT-2 effectively enhances Vietnamese community-based COVID-19 question answering, outperforming existing models and filling a research gap in applying GPT to Vietnamese NLP tasks.

Contribution

It introduces GPT-2 for Vietnamese COVID-19 question answering and compares its performance with other models, showing superior results.

Findings

GPT-2 outperforms other SOTA models in accuracy

GPT-2 achieves promising results on Vietnamese COVID-19 dataset

The approach addresses a research gap in Vietnamese NLP applications

Abstract

Recent studies have provided empirical evidence of the wide-ranging potential of Generative Pre-trained Transformer (GPT), a pretrained language model, in the field of natural language processing. GPT has been effectively employed as a decoder within state-of-the-art (SOTA) question answering systems, yielding exceptional performance across various tasks. However, the current research landscape concerning GPT's application in Vietnamese remains limited. This paper aims to address this gap by presenting an implementation of GPT-2 for community-based question answering specifically focused on COVID-19 related queries in Vietnamese. We introduce a novel approach by conducting a comparative analysis of different Transformers vs SOTA models in the community-based COVID-19 question answering dataset. The experimental findings demonstrate that the GPT-2 models exhibit highly promising outcomes, outperforming other SOTA models as well as previous community-based COVID-19 question answering models developed for Vietnamese.