SPOT: An Annotated French Corpus and Benchmark for Detecting Critical Interventions in Online Conversations

TL;DR

This paper introduces SPOT, a French corpus and benchmark for detecting critical interventions in online discussions, focusing on subtle social cues that influence conversation flow.

Contribution

It presents the first annotated corpus for detecting stopping points in French social media comments and benchmarks models for this novel NLP task.

Findings

Fine-tuned encoder models outperform prompted LLMs in F1 score.

Contextual metadata improves model performance.

Supervised learning is crucial for non-English social media analysis.

Abstract

We introduce SPOT (Stopping Points in Online Threads), the first annotated corpus translating the sociological concept of stopping point into a reproducible NLP task. Stopping points are ordinary critical interventions that pause or redirect online discussions through a range of forms (irony, subtle doubt or fragmentary arguments) that frameworks like counterspeech or social correction often overlook. We operationalize this concept as a binary classification task and provide reliable annotation guidelines. The corpus contains 43,305 manually annotated French Facebook comments linked to URLs flagged as false information by social media users, enriched with contextual metadata (article, post, parent comment, page or group, and source). We benchmark fine-tuned encoder models (CamemBERT) and instruction-tuned LLMs under various prompting strategies. Results show that fine-tuned encoders outperform prompted LLMs in F1 score by more than 10 percentage points, confirming the importance of supervised learning for emerging non-English social media tasks. Incorporating contextual metadata further improves encoder models F1 scores from 0.75 to 0.78. We release the anonymized dataset, along with the annotation guidelines and code in our code repository, to foster transparency and reproducible research.