Malicious ML Model Detection by Learning Dynamic Behaviors

TL;DR

This paper introduces DynaHug, a dynamic analysis and machine learning-based method for detecting malicious pre-trained ML models, outperforming existing static and heuristic detectors.

Contribution

The paper presents DynaHug, a novel dynamic behavior learning approach using one-class SVM to identify malicious models, addressing limitations of static analysis methods.

Findings

DynaHug achieves up to 44% higher F1-score than baseline detectors.

Dynamic analysis and clustering improve detection effectiveness.

Evaluation on over 25,000 models demonstrates robustness.

Abstract

Pre-trained machine learning models (PTMs) are commonly provided via Model Hubs (e.g., Hugging Face) in standard formats like Pickles to facilitate accessibility and reuse. However, this ML supply chain setting is susceptible to malicious attacks that are capable of executing arbitrary code on trusted user environments, e.g., during model loading. To detect malicious PTMs, state-of-the-art detectors (e.g., PickleScan) rely on rules, heuristics, or static analysis, but ignore runtime model behaviors. Consequently, they either miss malicious models due to under-approximation (blacklisting) or miscategorize benign models due to over-approximation (static analysis or whitelisting). To address this challenge, we propose a novel technique (DynaHug) which detects malicious PTMs by learning the behavior of benign PTMs using dynamic analysis and machine learning (ML). DynaHug trains an ML classifier (one-class SVM (OCSVM)) on the runtime behaviours of task-specific benign models. We evaluate DynaHug using over 25,000 benign and malicious PTMs from different sources including Hugging Face and MalHug. We also compare DynaHug to several state-of-the-art detectors including static, dynamic and LLM-based detectors. Results show that DynaHug is up to 44% more effective than existing baselines in terms of F1-score. Our ablation study demonstrates that our design decisions (dynamic analysis, OCSVM, clustering) contribute positively to DynaHug's effectiveness.