CLIP the Landscape: Automated Tagging of Crowdsourced Landscape Images

TL;DR

This paper introduces a CLIP-based multi-modal classifier for automatically tagging landscape images with geographical context, improving accuracy by combining location, title, and image features, and providing a lightweight, accessible pipeline.

Contribution

It presents a novel multi-modal approach that enhances landscape image tagging accuracy and releases an easy-to-use pipeline for spatial data enrichment.

Findings

Combining location and title embeddings improves tag prediction accuracy.

The pipeline trains on a modest laptop using pre-trained CLIP embeddings.

Predicted tags can enhance spatial understanding in data-sparse regions.

Abstract

We present a CLIP-based, multi-modal, multi-label classifier for predicting geographical context tags from landscape photos in the Geograph dataset--a crowdsourced image archive spanning the British Isles, including remote regions lacking POIs and street-level imagery. Our approach addresses a Kaggle competition\footnote{https://www.kaggle.com/competitions/predict-geographic-context-from-landscape-photos} task based on a subset of Geograph's 8M images, with strict evaluation: exact match accuracy is required across 49 possible tags. We show that combining location and title embeddings with image features improves accuracy over using image embeddings alone. We release a lightweight pipeline\footnote{https://github.com/SpaceTimeLab/ClipTheLandscape} that trains on a modest laptop, using pre-trained CLIP image and text embeddings and a simple classification head. Predicted tags can support downstream tasks such as building location embedders for GeoAI applications, enriching spatial understanding in data-sparse regions.