Towards Robust Detection of Open Source Software Supply Chain Poisoning Attacks in Industry Environments

TL;DR

This paper introduces OSCAR, a dynamic analysis pipeline for detecting open-source package poisoning attacks in industry environments, significantly reducing false positives and effectively identifying malicious packages in NPM and PyPI ecosystems.

Contribution

OSCAR is a novel dynamic detection system that fully executes packages, employs fuzz testing, and monitors behavior with API hooks, improving detection accuracy and reducing false positives in real-world industrial settings.

Findings

Achieves F1 scores of 0.95 in NPM and 0.91 in PyPI.

Reduces false positive rates by over 30% compared to existing tools.

Identified over 11,600 malicious packages in a real-world deployment.

Abstract

The exponential growth of open-source package ecosystems, particularly NPM and PyPI, has led to an alarming increase in software supply chain poisoning attacks. Existing static analysis methods struggle with high false positive rates and are easily thwarted by obfuscation and dynamic code execution techniques. While dynamic analysis approaches offer improvements, they often suffer from capturing non-package behaviors and employing simplistic testing strategies that fail to trigger sophisticated malicious behaviors. To address these challenges, we present OSCAR, a robust dynamic code poisoning detection pipeline for NPM and PyPI ecosystems. OSCAR fully executes packages in a sandbox environment, employs fuzz testing on exported functions and classes, and implements aspect-based behavior monitoring with tailored API hook points. We evaluate OSCAR against six existing tools using a comprehensive benchmark dataset of real-world malicious and benign packages. OSCAR achieves an F1 score of 0.95 in NPM and 0.91 in PyPI, confirming that OSCAR is as effective as the current state-of-the-art technologies. Furthermore, for benign packages exhibiting characteristics typical of malicious packages, OSCAR reduces the false positive rate by an average of 32.06% in NPM (from 34.63% to 2.57%) and 39.87% in PyPI (from 41.10% to 1.23%), compared to other tools, significantly reducing the workload of manual reviews in real-world deployments. In cooperation with Ant Group, a leading financial technology company, we have deployed OSCAR on its NPM and PyPI mirrors since January 2023, identifying 10,404 malicious NPM packages and 1,235 malicious PyPI packages over 18 months. This work not only bridges the gap between academic research and industrial application in code poisoning detection but also provides a robust and practical solution that has been thoroughly tested in a real-world industrial setting.