Moltbook Moderation: Uncovering Hidden Intent Through Multi-Turn Dialogue

TL;DR

This paper introduces BOT-MOD, a novel multi-turn dialogue framework that detects malicious agent intent in multi-agent systems by engaging in Gibbs-sampling guided exchanges, moving beyond content filtering.

Contribution

It presents a new intent-based moderation framework that effectively identifies malicious behaviors through multi-turn interactions, outperforming traditional content-based methods.

Findings

BOT-MOD reliably detects agent intent across various adversarial setups.

The framework maintains a low false positive rate on benign behaviors.

Constructed dataset from Moltbook enables comprehensive evaluation.

Abstract

The emergence of multi-agent systems introduces novel moderation challenges that extend beyond content filtering. Agents with malicious intent may contribute harmful content that appears benign to evade content-based moderation, while compromising the system through exploitative and malicious behavior manifested across their overall interaction patterns within the community. To address this, we introduce BOT-MOD (BOT-MODeration), a moderation framework that grounds detection in agent intent rather than traditional content level signals. BOT-MOD identifies the underlying intent by engaging with the target agent in a multi-turn exchange guided by Gibbs-based sampling over candidate intent hypotheses. This progressively narrows the space of plausible agent objectives to identify the underlying behavior. To evaluate our approach, we construct a dataset derived from Moltbook that encompasses diverse benign and malicious behaviors based on actual community structures, posts, and comments. Results demonstrate that BOT-MOD reliably identifies agent intent across a range of adversarial configurations, while maintaining a low false positive rate on benign behaviors. This work advances the foundation for scalable, intent-aware moderation of agents in open multi-agent environments.