Detecting Visual Triggers in Cannabis Imagery: A CLIP-Based Multi-Labeling Framework with Local-Global Aggregation

TL;DR

This paper presents a CLIP-based multi-labeling framework to analyze visual and textual features in online cannabis discussions, revealing how specific image and text attributes influence user engagement.

Contribution

The study introduces a novel CLIP-based multi-labeling approach combined with local-global aggregation for analyzing cannabis imagery and associated text in social media discussions.

Findings

Food-related visuals positively correlate with user engagement.

Certain textual topics like cannabis legalization increase engagement.

Image colorfulness and specific themes negatively impact user interaction.

Abstract

This study investigates the interplay of visual and textual features in online discussions about cannabis edibles and their impact on user engagement. Leveraging the CLIP model, we analyzed 42,743 images from Facebook (March 1 to August 31, 2021), with a focus on detecting food-related visuals and examining the influence of image attributes such as colorfulness and brightness on user interaction. For textual analysis, we utilized the BART model as a denoising autoencoder to classify ten topics derived from structural topic modeling, exploring their relationship with user engagement. Linear regression analysis identified significant positive correlations between food-related visuals (e.g., fruit, candy, and bakery) and user engagement scores, as well as between engagement and text topics such as cannabis legalization. In contrast, negative associations were observed with image colorfulness and certain textual themes. These findings offer actionable insights for policymakers and regulatory bodies in designing warning labels and marketing regulations to address potential risks associated with recreational cannabis edibles.