Unsupervised Multimodal Graph-based Model for Geo-social Analysis

TL;DR

This paper introduces an unsupervised, multimodal graph-based model that integrates semantic and geographic information for geo-social analysis, improving clustering and interpretability in disaster-related social media data.

Contribution

It presents a novel end-to-end framework with two architectures, MonoGrah and MultiGraph, for joint embedding of multimodal data into a unified space.

Findings

Outperforms baselines in topic quality and spatial coherence

Produces semantically coherent and spatially compact clusters

Demonstrates effectiveness on four real-world disaster datasets

Abstract

The systematic analysis of user-generated social media content, especially when enriched with geospatial context, plays a vital role in domains such as disaster management and public opinion monitoring. Although multimodal approaches have made significant progress, most existing models remain fragmented, processing each modality separately rather than integrating them into a unified end-to-end model. To address this, we propose an unsupervised, multimodal graph-based methodology that jointly embeds semantic and geographic information into a shared representation space. The proposed methodology comprises two architectural paradigms: a mono graph (MonoGrah) model that jointly encodes both modalities, and a multi graph (MultiGraph) model that separately models semantic and geographic relationships and subsequently integrates them through multi-head attention mechanisms. A composite loss, combining contrastive, coherence, and alignment objectives, guides the learning process to produce semantically coherent and spatially compact clusters. Experiments on four real-world disaster datasets demonstrate that our models consistently outperform existing baselines in topic quality, spatial coherence, and interpretability. Inherently domain-independent, the framework can be readily extended to diverse forms of multimodal data and a wide range of downstream analysis tasks.