Efficient RL-based Cache Vulnerability Exploration by Penalizing Useless Agent Actions

TL;DR

This paper introduces an improved RL-based method for exploring cache vulnerabilities by penalizing ineffective actions, significantly enhancing exploration efficiency and reducing training time.

Contribution

It proposes a novel approach to identify and penalize useless actions in RL training for cache vulnerability analysis, improving efficiency over naive methods.

Findings

Useless actions reduced by up to 43.08%.

Training time decreased by 28% in the base case.

Efficiency improvements observed across 17 cache structures.

Abstract

Cache-timing attacks exploit microarchitectural characteristics to leak sensitive data, posing a severe threat to modern systems. Despite its severity, analyzing the vulnerability of a given cache structure against cache-timing attacks is challenging. To this end, a method based on Reinforcement Learning (RL) has been proposed to automatically explore vulnerabilities for a given cache structure. However, a naive RL-based approach suffers from inefficiencies due to the agent performing actions that do not contribute to the exploration. In this paper, we propose a method to identify these useless actions during training and penalize them so that the agent avoids them and the exploration efficiency is improved. Experiments on 17 cache structures show that our training mechanism reduces the number of useless actions by up to 43.08%. This resulted in the reduction of training time by 28\% in the base case and 4.84\% in the geomean compared to a naive RL-based approach.