The Shadow of Fraud: The Emerging Danger of AI-powered Social Engineering and its Possible Cure

TL;DR

This paper surveys the rise of AI-powered social engineering attacks, categorizes their evolution into three phases, and discusses strategies for measuring and defending against these increasingly sophisticated threats.

Contribution

It introduces a new categorization of AI-driven social engineering attacks into 3E phases and highlights the need for a risk assessment framework to improve defenses.

Findings

AI enhances attack personalization and effectiveness

Three phases of attack evolution: Enlarging, Enriching, Emerging

Need for proactive defense strategies and risk assessment

Abstract

Social engineering (SE) attacks remain a significant threat to both individuals and organizations. The advancement of Artificial Intelligence (AI), including diffusion models and large language models (LLMs), has potentially intensified these threats by enabling more personalized and convincing attacks. This survey paper categorizes SE attack mechanisms, analyzes their evolution, and explores methods for measuring these threats. It highlights the challenges in raising awareness about the risks of AI-enhanced SE attacks and offers insights into developing proactive and adaptable defense strategies. Additionally, we introduce a categorization of the evolving nature of AI-powered social engineering attacks into "3E phases": Enlarging, wherein the magnitude of attacks expands through the leverage of digital media; Enriching, introducing novel attack vectors and techniques; and Emerging, signifying the advent of novel threats and methods. Moreover, we emphasize the necessity for a robust framework to assess the risk of AI-powered SE attacks. By identifying and addressing gaps in existing research, we aim to guide future studies and encourage the development of more effective defenses against the growing threat of AI-powered social engineering.