MATCHED: Multimodal Authorship-Attribution To Combat Human Trafficking in Escort-Advertisement Data

TL;DR

This paper introduces MATCHED, a multimodal dataset and benchmark for authorship attribution in escort ads, demonstrating that combining text and images improves detection of traffickers and enhances law enforcement capabilities.

Contribution

The study creates a large multimodal dataset and evaluates multimodal models, showing improved performance over text-only or vision-only approaches for identifying traffickers in escort ads.

Findings

Multimodal features outperform unimodal baselines in vendor identification.

Visual cues provide stylistic information that complements textual analysis.

End-to-end multimodal training is more effective than separate or alignment-based methods.

Abstract

Human trafficking (HT) remains a critical issue, with traffickers increasingly leveraging online escort advertisements (ads) to advertise victims anonymously. Existing detection methods, including Authorship Attribution (AA), often center on text-based analyses and neglect the multimodal nature of online escort ads, which typically pair text with images. To address this gap, we introduce MATCHED, a multimodal dataset of 27,619 unique text descriptions and 55,115 unique images collected from the Backpage escort platform across seven U.S. cities in four geographical regions. Our study extensively benchmarks text-only, vision-only, and multimodal baselines for vendor identification and verification tasks, employing multitask (joint) training objectives that achieve superior classification and retrieval performance on in-distribution and out-of-distribution (OOD) datasets. Integrating multimodal features further enhances this performance, capturing complementary patterns across text and images. While text remains the dominant modality, visual data adds stylistic cues that enrich model performance. Moreover, text-image alignment strategies like CLIP and BLIP2 struggle due to low semantic overlap and vague connections between the modalities of escort ads, with end-to-end multimodal training proving more robust. Our findings emphasize the potential of multimodal AA (MAA) to combat HT, providing LEAs with robust tools to link ads and disrupt trafficking networks.