Context-Aware Spear Phishing: Generative AI-Enabled Attacks Against Individuals via Public Social Media Data

TL;DR

This paper demonstrates how generative AI and social media data can be exploited to create highly personalized spear-phishing attacks, highlighting risks and evaluating defenses.

Contribution

It introduces a modular framework for automated, context-aware phishing attacks using GenAI, and provides large-scale evaluation and analysis of defense mechanisms.

Findings

GenAI-produced emails are more personalized and persuasive than real-world phishing messages.

User study shows GenAI attacks are less suspicious and more effective.

Existing defenses are insufficient against context-aware, adaptive AI-generated attacks.

Abstract

We demonstrate how publicly available social-media data and generative AI (GenAI) can be misused to automate and scale highly personalized, context-aware spear-phishing campaigns. With minimal attacker effort, a small amount of public activity per target is sufficient for GenAI models to extract interests and contextual cues, producing persuasive messages that mirror a target's style while bypassing generic content-moderation safeguards. We introduce a modular framework that combines multimodal signal extraction, communication-style profiling, and attack-type instantiation across seven strategies (baiting, scareware, honey trap, tailgating, impersonation, quid pro quo, and personalized emotional exploitation). We conduct a large-scale, multi-model evaluation covering thousands of generated emails and eight security-relevant criteria, benchmarking against a corpus of real-world phishing messages. The GenAI-produced emails exhibit markedly higher personalization, contextual grounding, and persuasive leverage. Importantly, a complementary user study corroborates these results, revealing that LLM-generated attacks consistently outperform APWG eCrimeX emails across eight dimensions while eliciting lower suspicion among human recipients. Finally, we measure and analyze the behavior of existing proactive, prompt-level defense mechanisms, which incorporate adaptive mechanisms, as well as two complementary defense approaches-policy-augmented SOTA safeguard models and system-instruction chain-of-thought moderation. We document how these defenses respond to contextualized and adaptive attack prompts, underscoring the need for platform-level safeguards that explicitly account for contextualized abuse at scale.