Social-Media Based Personas Challenge: Hybrid Prediction of Common and Rare User Actions on Bluesky

TL;DR

This paper introduces a hybrid approach for predicting both common and rare user actions on social media, using diverse models and features, validated on a large Bluesky dataset, and winning a related challenge.

Contribution

It presents a novel hybrid methodology combining multiple models and features to predict diverse user actions, including rare behaviors, on social media platforms.

Findings

Achieved macro F1-score of 0.64 for common actions.

Achieved macro F1-score of 0.56 for rare actions.

First place in the SocialSim challenge at COLM 2025.

Abstract

Understanding and predicting user behavior on social media platforms is crucial for content recommendation and platform design. While existing approaches focus primarily on common actions like retweeting and liking, the prediction of rare but significant behaviors remains largely unexplored. This paper presents a hybrid methodology for social media user behavior prediction that addresses both frequent and infrequent actions across a diverse action vocabulary. We evaluate our approach on a large-scale Bluesky dataset containing 6.4 million conversation threads spanning 12 distinct user actions across 25 persona clusters. Our methodology combines four complementary approaches: (i) a lookup database system based on historical response patterns; (ii) persona-specific LightGBM models with engineered temporal and semantic features for common actions; (iii) a specialized hybrid neural architecture fusing textual and temporal representations for rare action classification; and (iv) generation of text replies. Our persona-specific models achieve an average macro F1-score of 0.64 for common action prediction, while our rare action classifier achieves 0.56 macro F1-score across 10 rare actions. These results demonstrate that effective social media behavior prediction requires tailored modeling strategies recognizing fundamental differences between action types. Our approach achieved first place in the SocialSim: Social-Media Based Personas challenge organized at the Social Simulation with LLMs workshop at COLM 2025.