An Empirical Study of Safetensors' Usage Trends and Developers' Perceptions

TL;DR

This study analyzes the adoption and perceptions of safetensors serialization among ML developers on Hugging Face, revealing increased usage, automation in conversions, and mixed developer reactions to the new format.

Contribution

It provides empirical insights into how ML developers are adopting safetensors, highlighting trends, challenges, and attitudes towards this safer serialization method.

Findings

Increased adoption of safetensors among developers.

Many conversions are automated via Hugging Face tools.

Developers face issues but show willingness to learn.

Abstract

Developers are sharing pre-trained Machine Learning (ML) models through a variety of model sharing platforms, such as Hugging Face, in an effort to make ML development more collaborative. To share the models, they must first be serialized. While there are many methods of serialization in Python, most of them are unsafe. To tame this insecurity, Hugging Face released safetensors as a way to mitigate the threats posed by unsafe serialization formats. In this context, this paper investigates developer's shifts towards using safetensors on Hugging Face in an effort to understand security practices in the ML development community, as well as how developers react to new methods of serialization. Our results find that more developers are adopting safetensors, and many safetensor adoptions were made by automated conversions of existing models by Hugging Face's conversion tool. We also found, however, that a majority of developers ignore the conversion tool's pull requests, and that while many developers are facing issues with using safetensors, they are eager to learn about and adapt the format.