Adaptive Network Security Policies via Belief Aggregation and Rollout

TL;DR

This paper introduces a scalable, theoretically guaranteed method for adaptive network security policy management using belief estimation, feature-based aggregation, and rollout, which quickly adapts to system changes.

Contribution

The paper presents a novel approach combining belief estimation, offline feature-based aggregation, and online rollout for scalable, adaptive security policy computation with performance guarantees.

Findings

Outperforms state-of-the-art methods on benchmarks including CAGE-2.

Provides theoretical guarantees and efficient adaptation to system changes.

Demonstrates effectiveness through simulations and testbed results.

Abstract

Evolving security vulnerabilities and shifting operational conditions require frequent updates to network security policies. These updates include adjustments to incident response procedures and modifications to access controls, among others. Reinforcement learning methods have been proposed for automating such policy adaptations, but most methods in the research literature lack performance guarantees and adapt slowly to changes. In this paper, we address these limitations and present a method for computing security policies that is scalable, offers theoretical guarantees, and adapts quickly to changes. The method uses a model or simulator of the system, which is updated when changes occur, and combines three components: belief estimation through particle filtering, offline policy computation through feature-based aggregation, and online policy adaptation through rollout. In particular, feature-based aggregation enables scalable offline optimization of a policy, while rollout adapts the policy online to changes in the system model without repeating the offline optimization. We analyze the approximation error of the aggregation and show that the rollout efficiently adapts policies to changes under certain conditions. Simulations and testbed results demonstrate that our method outperforms state-of-the-art methods on several benchmarks, including CAGE-2.