Post-Training Local LLM Agents for Linux Privilege Escalation with Verifiable Rewards

TL;DR

This paper introduces a two-stage post-training method for small, local LLMs to effectively perform Linux privilege escalation, achieving high success rates with verifiable rewards and significantly reduced inference costs.

Contribution

It presents a novel two-stage post-training pipeline combining supervised fine-tuning and reinforcement learning for security tasks on small models.

Findings

Supervised fine-tuning more than doubles baseline success rate.

Reinforcement learning boosts success rate to 95.8%.

Inference cost per successful escalation reduced by over 100x.

Abstract

LLM agents are increasingly relevant to research domains such as vulnerability discovery. Yet, the strongest systems remain closed and cloud-only, making them resource-intensive, difficult to reproduce, and unsuitable for work involving proprietary code or sensitive data. Consequently, there is an urgent need for small, local models that can perform security tasks under strict resource budgets, but methods for developing them remain underexplored. In this paper, we address this gap by proposing a two-stage post-training pipeline. We focus on the problem of Linux privilege escalation, where success is automatically verifiable and the task requires multi-step interactive reasoning. Using an experimental setup that prevents data leakage, we post-train a 4B model in two stages: supervised fine-tuning on traces from procedurally generated privilege-escalation environments, followed by reinforcement learning with verifiable rewards. On a held-out benchmark of 12 Linux privilege-escalation scenarios, supervised fine-tuning alone more than doubles the baseline success rate at 20 rounds, and reinforcement learning further lifts our resulting model, PrivEsc-LLM, to 95.8%, nearly matching Claude Opus 4.6 at 97.5%. At the same time, the expected inference cost per successful escalation is reduced by over 100x.