The End of Trust: How Agentic AI Breaks Security Assumptions

TL;DR

Agentic AI fundamentally alters digital security by enabling high-fidelity, mass-scale deception, challenging existing detection and verification paradigms and requiring new governance approaches.

Contribution

The paper introduces the Infinite Impostor attack model and proposes a shift from actor authentication to action evaluation for security.

Findings

Agentic AI enables scalable, personalized deception at mass scale.

Detection systems assuming synthetic outputs are distinguishable are likely to fail.

Governance tensions increase as platforms become regulators of digital interactions.

Abstract

For decades, the security of digital interaction has rested on an unacknowledged economic constraint. Attackers faced a tradeoff between the fidelity of a deception and the scale at which it could be deployed. Convincing impersonation required sustained human effort and was confined to a narrow set of high-value targets, while mass-market attacks sacrificed plausibility for reach. Detection systems, verification mechanisms, and user awareness training have all been implicitly calibrated to the artifacts of cheap deception that this tradeoff produced. Agentic AI collapses the tradeoff, allowing high-fidelity, individually tailored deception to be produced at mass-market scale. We argue that this shift exhausts a security paradigm rather than merely intensifying the threat landscape. We introduce the Infinite Impostor, an attack model in which an autonomous agent interposes itself between two parties who already trust each other, hijacking an existing relationship rather than building a new one from scratch. Detection-oriented defenses share an assumption that generative progress is eliminating, that synthetic outputs are distinguishable from authentic ones. We propose a suspect-by-default paradigm that shifts security from authenticating actors to evaluating actions, and examine the governance tensions that arise when platforms become the regulatory substrate of digital interaction.