Robust Image Captioning

TL;DR

This paper introduces a robust image captioning method that uses object relation graphs and adversarial training to improve attention mechanisms, demonstrating promising experimental results.

Contribution

It proposes a novel approach combining object relation graphs with adversarial robust cut algorithms for enhanced image captioning.

Findings

Demonstrates improved captioning performance on benchmark datasets.

Shows robustness of the method against adversarial perturbations.

Highlights the importance of spatial object relations in captioning accuracy.

Abstract

Automated captioning of photos is a mission that incorporates the difficulties of photo analysis and text generation. One essential feature of captioning is the concept of attention: how to determine what to specify and in which sequence. In this study, we leverage the Object Relation using adversarial robust cut algorithm, that grows upon this method by specifically embedding knowledge about the spatial association between input data through graph representation. Our experimental study represent the promising performance of our proposed method for image captioning.