# A customisable pipeline for continuously harvesting socially-minded Twitter users

## TL;DR

This paper presents a customizable pipeline for continuously collecting and analyzing Twitter user data within localized social contexts, aiding the identification of socially-minded users like activists through network and content metrics.

## Contribution

It introduces a novel, adaptable pipeline for real-time Twitter user profiling based on localized social contexts, facilitating the study of online activism.

## Key findings

- Pipeline successfully identified social activists in healthcare campaigns
- Experimental ranking functions differentiated user types effectively
- Code implementation is publicly available for replication

## Abstract

On social media platforms and Twitter in particular, specific classes of users such as influencers have been given satisfactory operational definitions in terms of network and content metrics. Others, for instance online activists, are not less important but their characterisation still requires experimenting. We make the hypothesis that such interesting users can be found within temporally and spatially localised contexts, i.e., small but topical fragments of the network containing interactions about social events or campaigns with a significant footprint on Twitter. To explore this hypothesis, we have designed a continuous user profile discovery pipeline that produces an ever-growing dataset of user profiles by harvesting and analysing contexts from the Twitter stream. The profiles dataset includes key network and content-based users metrics, enabling experimentation with user-defined score functions that characterise specific classes of online users. The paper describes the design and implementation of the pipeline and its empirical evaluation on a case study consisting of healthcare-related campaigns in the UK, showing how it supports the operational definitions of online activism, by comparing three experimental ranking functions. The code is publicly available.