One RNG to Rule Them All: How Randomness Becomes an Attack Vector in Machine Learning

TL;DR

This paper investigates the security vulnerabilities arising from the use of pseudorandom number generators in machine learning systems, highlighting potential attack vectors and proposing a static and runtime defense tool called RNGGuard.

Contribution

It introduces RNGGuard, a tool that analyzes and enforces secure randomness in machine learning frameworks to prevent adversarial exploitation of PRNGs.

Findings

RNGGuard effectively identifies insecure random functions in ML code.

RNGGuard enforces secure execution of random functions at runtime.

The approach closes security gaps in current ML systems' randomness sources.

Abstract

Machine learning relies on randomness as a fundamental component in various steps such as data sampling, data augmentation, weight initialization, and optimization. Most machine learning frameworks use pseudorandom number generators as the source of randomness. However, variations in design choices and implementations across different frameworks, software dependencies, and hardware backends along with the lack of statistical validation can lead to previously unexplored attack vectors on machine learning systems. Such attacks on randomness sources can be extremely covert, and have a history of exploitation in real-world systems. In this work, we examine the role of randomness in the machine learning development pipeline from an adversarial point of view, and analyze the implementations of PRNGs in major machine learning frameworks. We present RNGGuard to help machine learning engineers secure their systems with low effort. RNGGuard statically analyzes a target library's source code and identifies instances of random functions and modules that use them. At runtime, RNGGuard enforces secure execution of random functions by replacing insecure function calls with RNGGuard's implementations that meet security specifications. Our evaluations show that RNGGuard presents a practical approach to close existing gaps in securing randomness sources in machine learning systems.