MF2-MVQA: A Multi-stage Feature Fusion method for Medical Visual Question Answering

TL;DR

MF2-MVQA introduces a stage-wise feature fusion approach for medical visual question answering, effectively combining multi-level visual features with text to improve performance on medical VQA datasets.

Contribution

It proposes a novel multi-stage feature fusion method that better utilizes multi-scale visual information in medical VQA tasks, outperforming previous methods.

Findings

Achieved state-of-the-art results on VQA-Med 2019 and VQA-RAD datasets.

Visualization confirms the effectiveness of the feature fusion approach.

Outperforms previous methods in medical visual question answering.

Abstract

There is a key problem in the medical visual question answering task that how to effectively realize the feature fusion of language and medical images with limited datasets. In order to better utilize multi-scale information of medical images, previous methods directly embed the multi-stage visual feature maps as tokens of same size respectively and fuse them with text representation. However, this will cause the confusion of visual features at different stages. To this end, we propose a simple but powerful multi-stage feature fusion method, MF2-MVQA, which stage-wise fuses multi-level visual features with textual semantics. MF2-MVQA achieves the State-Of-The-Art performance on VQA-Med 2019 and VQA-RAD dataset. The results of visualization also verify that our model outperforms previous work.