Cybersecurity AI: The Dangerous Gap Between Automation and Autonomy

TL;DR

This paper clarifies the distinction between automation and autonomy in cybersecurity AI, proposing a taxonomy to prevent misconceptions and promote effective human-AI collaboration for safer deployment.

Contribution

It introduces a 6-level taxonomy distinguishing automation from autonomy in cybersecurity AI, highlighting current capabilities and emphasizing the need for clear terminology and transparency.

Findings

Current autonomous pentesters operate at Level 3-4, requiring human oversight.

Mischaracterizing AI as fully autonomous can reduce necessary human oversight.

True Level 5 autonomy in cybersecurity AI remains an aspirational goal.

Abstract

The cybersecurity industry combines "automated" and "autonomous" AI, creating dangerous misconceptions about system capabilities. Recent milestones like XBOW topping HackerOne's leaderboard showcase impressive progress, yet these systems remain fundamentally semi-autonomous--requiring human oversight. Drawing from robotics principles, where the distinction between automation and autonomy is well-established, I take inspiration from prior work and establish a 6-level taxonomy (Level 0-5) distinguishing automation from autonomy in Cybersecurity AI. Current "autonomous" pentesters operate at Level 3-4: they execute complex attack sequences but need human review for edge cases and strategic decisions. True Level 5 autonomy remains aspirational. Organizations deploying mischaracterized "autonomous" tools risk reducing oversight precisely when it's most needed, potentially creating new vulnerabilities. The path forward requires precise terminology, transparent capabilities disclosure, and human-AI partnership-not replacement.