Dynamic Risk Assessments for Offensive Cybersecurity Agents

TL;DR

This paper emphasizes the importance of dynamic risk assessments for offensive cybersecurity agents, demonstrating that adversaries can significantly improve agent capabilities within limited compute budgets, which current audits often overlook.

Contribution

It introduces an expanded threat model considering adversaries' degrees of freedom and demonstrates the potential for substantial capability improvements through iterative optimization.

Findings

Adversaries can improve cybersecurity capabilities by over 40% within 8 GPU hours.

Current audits often underestimate risks by not accounting for iterative adversarial improvements.

Dynamic risk evaluation provides a more accurate picture of potential threats.

Abstract

Foundation models are increasingly becoming better autonomous programmers, raising the prospect that they could also automate dangerous offensive cyber-operations. Current frontier model audits probe the cybersecurity risks of such agents, but most fail to account for the degrees of freedom available to adversaries in the real world. In particular, with strong verifiers and financial incentives, agents for offensive cybersecurity are amenable to iterative improvement by would-be adversaries. We argue that assessments should take into account an expanded threat model in the context of cybersecurity, emphasizing the varying degrees of freedom that an adversary may possess in stateful and non-stateful environments within a fixed compute budget. We show that even with a relatively small compute budget (8 H100 GPU Hours in our study), adversaries can improve an agent's cybersecurity capability on InterCode CTF by more than 40\% relative to the baseline -- without any external assistance. These results highlight the need to evaluate agents' cybersecurity risk in a dynamic manner, painting a more representative picture of risk.