Upgrading the Newsroom: An Automated Image Selection System for News Articles
Fangyu Liu, R\'emi Lebret, Didier Orel, Philippe Sordet, Karl Aberer

TL;DR
This paper introduces an automated, multilingual image selection system for news articles that leverages hierarchical self-attention and char-level embeddings, improving retrieval accuracy over existing methods.
Contribution
The novel system combines multiple textual sources and multilingual data with advanced attention mechanisms to enhance news image selection accuracy.
Findings
Outperforms existing text-image retrieval methods in a multilingual news dataset.
Utilizes multiple textual sources to improve image relevance.
Shows advantages of multilingual and multi-source textual data in retrieval tasks.
Abstract
We propose an automated image selection system to assist photo editors in selecting suitable images for news articles. The system fuses multiple textual sources extracted from news articles and accepts multilingual inputs. It is equipped with char-level word embeddings to help both modeling morphologically rich languages, e.g. German, and transferring knowledge across nearby languages. The text encoder adopts a hierarchical self-attention mechanism to attend more to both keywords within a piece of text and informative components of a news article. We extensively experiment with our system on a large-scale text-image database containing multimodal multilingual news articles collected from Swiss local news media websites. The system is compared with multiple baselines with ablation studies and is shown to beat existing text-image retrieval methods in a weakly-supervised learning setting.…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsMultimodal Machine Learning Applications · Advanced Image and Video Retrieval Techniques · Image Retrieval and Classification Techniques
