Journalistic Guidelines Aware News Image Captioning

TL;DR

This paper introduces JoGANIC, a novel news image captioning method that incorporates journalistic guidelines and named entity awareness, significantly improving caption quality and relevance compared to existing approaches.

Contribution

The paper presents a new approach, JoGANIC, which integrates journalistic captioning guidelines and context from news articles to enhance image captioning performance.

Findings

JoGANIC outperforms state-of-the-art methods on large-scale datasets.

Incorporating journalistic guidelines improves caption relevance.

Named entity awareness enhances caption informativeness.

Abstract

The task of news article image captioning aims to generate descriptive and informative captions for news article images. Unlike conventional image captions that simply describe the content of the image in general terms, news image captions follow journalistic guidelines and rely heavily on named entities to describe the image content, often drawing context from the whole article they are associated with. In this work, we propose a new approach to this task, motivated by caption guidelines that journalists follow. Our approach, Journalistic Guidelines Aware News Image Captioning (JoGANIC), leverages the structure of captions to improve the generation quality and guide our representation design. Experimental results, including detailed ablation studies, on two large-scale publicly available datasets show that JoGANIC substantially outperforms state-of-the-art methods both on caption generation and named entity related metrics.