PickleFuzzer: A Case Study in Fuzzing for Discrepancies Between Python Pickle Implementations

TL;DR

PickleFuzzer is a custom fuzzing tool that detects discrepancies between Python's pickle implementations, uncovering security vulnerabilities and inconsistencies that can undermine untrusted data handling.

Contribution

We developed PickleFuzzer, a generation-based fuzzer that identifies implementation discrepancies without relying on a formal specification, revealing critical security issues.

Findings

Detected 14 discrepancies between pickle implementations.

Identified 4 critical discrepancies that can bypass security tools.

Disclosed security issues to Python Software Foundation and bug bounty platform.

Abstract

Python's native serialization protocol, pickle, is a powerful but insecure format for transferring untrusted data. It is frequently used, especially for saving machine learning models, despite known security challenges. While developers sometimes mitigate this risk by restricting imports during unpickling or using static and dynamic analysis tools, these approaches are error-prone and depend heavily on accurate interpretations of the Pickle Virtual Machine (PVM) opcodes. Discrepancies across Python's three native PVM modules can lead to incorrect detection of malicious payloads and undermine existing defenses. To efficiently and scalably identify discrepancies, we present PickleFuzzer, a custom generation-based fuzzer that identifies inconsistencies across pickle implementations. PickleFuzzer generates pickle objects, passes them to each implementation, and detects differences in thrown exceptions or changes to key internal states. It generates pickle objects using a grammar, which we developed to account for the missing pickle specification. It determines discrepancies by comparing the execution behaviors of each test implementation, rather than requiring a specification-derived oracle. PickleFuzzer detected 14 new discrepancies between the pickle implementations. Four discrepancies are critical and can be used to bypass security-critical scanning tools like those deployed on the popular model hosting platform, Hugging Face. We disclosed all findings to the Python Software Foundation for remediation, and additionally disclosed the security issues to a bug bounty platform and were awarded a $750 bounty. We demonstrate that differential testing is a viable approach for identifying security-relevant discrepancies in important pickle implementations, and our work can lead to promising future directions for finding deeper pickle bugs with more directed fuzzing.