A Large-Scale Exploit Instrumentation Study of AI/ML Supply Chain Attacks in Hugging Face Models

TL;DR

This study examines the widespread use of unsafe serialization methods in Hugging Face models, demonstrating how these vulnerabilities can be exploited and assessing the platform's ability to detect such risks.

Contribution

It provides the first large-scale analysis of unsafe serialization in Hugging Face models and develops techniques to identify malicious or vulnerable models.

Findings

Many models use unsafe serialization methods

Hugging Face has limited detection of vulnerable models

Exploitation of unsafe serialization can compromise ML environments

Abstract

The development of machine learning (ML) techniques has led to ample opportunities for developers to develop and deploy their own models. Hugging Face serves as an open source platform where developers can share and download other models in an effort to make ML development more collaborative. In order for models to be shared, they first need to be serialized. Certain Python serialization methods are considered unsafe, as they are vulnerable to object injection. This paper investigates the pervasiveness of these unsafe serialization methods across Hugging Face, and demonstrates through an exploitation approach, that models using unsafe serialization methods can be exploited and shared, creating an unsafe environment for ML developers. We investigate to what extent Hugging Face is able to flag repositories and files using unsafe serialization methods, and develop a technique to detect malicious models. Our results show that Hugging Face is home to a wide range of potentially vulnerable models.