A TextGCN-Based Decoding Approach for Improving Remote Sensing Image Captioning

TL;DR

This paper introduces a novel TextGCN-based encoder-decoder framework for automatic remote sensing image captioning, significantly improving caption quality over existing methods through semantic embeddings and a fair search strategy.

Contribution

The paper presents a new TextGCN-enhanced encoder-decoder model combined with a comparison-based beam search for improved remote sensing image captioning.

Findings

Outperforms state-of-the-art encoder-decoder models on three datasets.

Achieves higher scores across BLEU, METEOR, ROUGE-L, and CIDEr metrics.

Demonstrates the effectiveness of semantic embeddings in image captioning.

Abstract

Remote sensing images are highly valued for their ability to address complex real-world issues such as risk management, security, and meteorology. However, manually captioning these images is challenging and requires specialized knowledge across various domains. This letter presents an approach for automatically describing (captioning) remote sensing images. We propose a novel encoder-decoder setup that deploys a Text Graph Convolutional Network (TextGCN) and multi-layer LSTMs. The embeddings generated by TextGCN enhance the decoder's understanding by capturing the semantic relationships among words at both the sentence and corpus levels. Furthermore, we advance our approach with a comparison-based beam search method to ensure fairness in the search strategy for generating the final caption. We present an extensive evaluation of our approach against various other state-of-the-art encoder-decoder frameworks. We evaluated our method across three datasets using seven metrics: BLEU-1 to BLEU-4, METEOR, ROUGE-L, and CIDEr. The results demonstrate that our approach significantly outperforms other state-of-the-art encoder-decoder methods.