LiT-4-RSVQA: Lightweight Transformer-based Visual Question Answering in Remote Sensing

TL;DR

This paper introduces LiT-4-RSVQA, a lightweight transformer-based model for remote sensing visual question answering that achieves accurate results with reduced computational resources, enabling practical deployment.

Contribution

The paper presents a novel lightweight transformer architecture for remote sensing VQA, focusing on efficiency without sacrificing accuracy.

Findings

Achieves accurate VQA results on benchmark datasets.

Reduces computational requirements significantly.

Provides publicly available code for reproducibility.

Abstract

Visual question answering (VQA) methods in remote sensing (RS) aim to answer natural language questions with respect to an RS image. Most of the existing methods require a large amount of computational resources, which limits their application in operational scenarios in RS. To address this issue, in this paper we present an effective lightweight transformer-based VQA in RS (LiT-4-RSVQA) architecture for efficient and accurate VQA in RS. Our architecture consists of: i) a lightweight text encoder module; ii) a lightweight image encoder module; iii) a fusion module; and iv) a classification module. The experimental results obtained on a VQA benchmark dataset demonstrate that our proposed LiT-4-RSVQA architecture provides accurate VQA results while significantly reducing the computational requirements on the executing hardware. Our code is publicly available at https://git.tu-berlin.de/rsim/lit4rsvqa.