CryptoTensors: A Light-Weight Large Language Model File Format for Highly-Secure Model Distribution
Huifeng Zhu, Shijie Li, Qinfeng Li, Yier Jin

TL;DR
CryptoTensors is a secure, lightweight file format extension for confidential large language model distribution, incorporating tensor encryption and access control while maintaining compatibility with existing frameworks.
Contribution
It introduces CryptoTensors, a novel extension to Safetensors that enables secure, efficient, and compatible distribution of confidential LLMs with minimal overhead.
Findings
Supports tensor-level encryption and access control
Maintains compatibility with Hugging Face and vLLM
Demonstrates minimal performance overhead
Abstract
To enhance the performance of large language models (LLMs) in various domain-specific applications, sensitive data such as healthcare, law, and finance are being used to privately customize or fine-tune these models. Such privately adapted LLMs are regarded as either personal privacy assets or corporate intellectual property. Therefore, protecting model weights and maintaining strict confidentiality during deployment and distribution have become critically important. However, existing model formats and deployment frameworks provide little to no built-in support for confidentiality, access control, or secure integration with trusted hardware. Current methods for securing model deployment either rely on computationally expensive cryptographic techniques or tightly controlled private infrastructure. Although these approaches can be effective in specific scenarios, they are difficult and…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsSecurity and Verification in Computing · Cryptography and Data Security · Scientific Computing and Data Management
