Location Sensitive Image Retrieval and Tagging

TL;DR

This paper introduces LocSens, a model that enhances image retrieval and tagging by integrating geographic location information, improving relevance and accuracy across different regions.

Contribution

The work presents a novel model that combines textual and location data for image retrieval and tagging, with new training strategies to balance location influence.

Findings

LocSens effectively ranks images based on location and tags.

The model improves image tagging accuracy using location data.

LocSens adapts to different levels of geographic granularity.

Abstract

People from different parts of the globe describe objects and concepts in distinct manners. Visual appearance can thus vary across different geographic locations, which makes location a relevant contextual information when analysing visual data. In this work, we address the task of image retrieval related to a given tag conditioned on a certain location on Earth. We present LocSens, a model that learns to rank triplets of images, tags and coordinates by plausibility, and two training strategies to balance the location influence in the final ranking. LocSens learns to fuse textual and location information of multimodal queries to retrieve related images at different levels of location granularity, and successfully utilizes location information to improve image tagging.