PhishDebate: An LLM-Based Multi-Agent Framework for Phishing Website Detection

TL;DR

PhishDebate introduces a multi-agent LLM framework that enhances phishing website detection by enabling specialized analysis, structured debate, and transparent reasoning, leading to higher accuracy and interpretability.

Contribution

It presents a novel multi-agent debate system for phishing detection that improves accuracy, interpretability, and scalability over existing single-agent approaches.

Findings

Achieves 98.2% recall on real-world phishing dataset.

Outperforms single-agent and Chain-of-Thought baselines.

Modular design allows customization and scalability.

Abstract

Phishing websites remain a major cybersecurity threat, exploiting deceptive structures, brand impersonation, and social engineering to evade detection. Recent advances in large language models (LLMs) have improved phishing detection through contextual understanding, yet most existing approaches rely on single-agent classification, which is prone to hallucination and often lacks interpretability and robustness. To address these limitations, we propose PhishDebate, a modular multi-agent LLM-based debate framework for phishing website detection. Four specialized agents independently analyze webpage aspects, including URL structure, HTML composition, semantic content, and brand impersonation, under the coordination of a Moderator and final Judge. Through structured debate and divergent reasoning, the framework achieves more accurate and interpretable decisions. By reducing uncertain predictions and providing transparent reasoning, PhishDebate functions as an analyst-augmentation system that lowers cognitive load and supports early, left-of-exploit detection of phishing threats. Evaluations on commercial LLMs show that PhishDebate achieves 98.2 % recall on a real-world phishing dataset and outperforms single-agent and Chain-of-Thought (CoT) baselines. Its modular design enables agent-level configurability, allowing adaptation to varying resource and application requirements, and offers scalability to high-velocity, large-scale security data environments.