PAT: Parallel Attention Transformer for Visual Question Answering in Vietnamese

TL;DR

This paper introduces the Parallel Attention Transformer (PAT) for Vietnamese visual question answering, utilizing novel modules like the Hierarchical Linguistic Features Extractor to improve accuracy on benchmark datasets.

Contribution

The paper proposes the PAT model with a new Parallel Attention mechanism and a Hierarchical Linguistic Features Extractor tailored for Vietnamese VQA tasks.

Findings

Achieved state-of-the-art accuracy on ViVQA dataset

Outperformed baseline models including SAAA and MCAN

Demonstrated effectiveness of the proposed modules

Abstract

We present in this paper a novel scheme for multimodal learning named the Parallel Attention mechanism. In addition, to take into account the advantages of grammar and context in Vietnamese, we propose the Hierarchical Linguistic Features Extractor instead of using an LSTM network to extract linguistic features. Based on these two novel modules, we introduce the Parallel Attention Transformer (PAT), achieving the best accuracy compared to all baselines on the benchmark ViVQA dataset and other SOTA methods including SAAA and MCAN.