Cybersecurity AI in OT: Insights from an AI Top-10 Ranker in the Dragos OT CTF 2025

TL;DR

This paper evaluates the performance of a cybersecurity AI during an industrial control system competition, demonstrating that AI can match or surpass human teams in early incident response phases, with insights into its capabilities and limitations.

Contribution

It provides the first detailed performance analysis of a cybersecurity AI in a large-scale OT CTF, highlighting its competitive edge and operational constraints.

Findings

AI reached top rank within 8 hours

AI scored 18,900 points, ranking 6th overall

AI outperformed human teams in early-phase incident response

Abstract

Operational Technology (OT) cybersecurity increasingly relies on rapid response across malware analysis, network forensics, and reverse engineering disciplines. We examine the performance of Cybersecurity AI (CAI), powered by the \texttt{alias1} model, during the Dragos OT CTF 2025 -- a 48-hour industrial control system (ICS) competition with more than 1,000 teams. Using CAI telemetry and official leaderboard data, we quantify CAI's trajectory relative to the leading human-operated teams. CAI reached Rank~1 between competition hours 7.0 and 8.0, crossed 10,000 points at 5.42~hours (1,846~pts/h), and completed 32 of the competition's 34 challenges before automated operations were paused at hour~24 with a final score of 18,900 points (6th place). The top-3 human teams solved 33 of 34 challenges, collectively leaving only the 600-point ``Kiddy Tags -- 1'' unsolved; they were also the only teams to clear the 1,000-point ``Moot Force'' binary. The top-5 human teams averaged 1,347~pts/h to the same milestone, marking a 37\% velocity advantage for CAI. We analyse time-resolved scoring, category coverage, and solve cadence. The evidence indicates that a mission-configured AI agent can match or exceed expert human crews in early-phase OT incident response while remaining subject to practical limits in sustained, multi-day operations.