VcLLM: Video Codecs are Secretly Tensor Codecs
Ceyu Xu, Yongji Wu, Xinyu Yang, Beidi Chen, Matthew Lentz, Danyang, Zhuo, Lisa Wu Wills

TL;DR
This paper reveals that video codecs can be repurposed as highly efficient tensor codecs, significantly reducing memory and bandwidth needs for large language model training and inference on consumer GPUs.
Contribution
It introduces a novel framework that uses video codecs as general-purpose tensor codecs, achieving state-of-the-art compression efficiency for LLMs.
Findings
Video codecs outperform traditional tensor compression methods.
The framework enables training large models on consumer-grade GPUs.
Significant reduction in memory and communication bandwidth requirements.
Abstract
As the parameter size of large language models (LLMs) continues to expand, the need for a large memory footprint and high communication bandwidth have become significant bottlenecks for the training and inference of LLMs. To mitigate these bottlenecks, various tensor compression techniques have been proposed to reduce the data size, thereby alleviating memory requirements and communication pressure. Our research found that video codecs, despite being originally designed for compressing videos, show excellent efficiency when compressing various types of tensors. We demonstrate that video codecs can be versatile and general-purpose tensor codecs while achieving the state-of-the-art compression efficiency in various tasks. We further make use of the hardware video encoding and decoding module available on GPUs to create a framework capable of both inference and training with video codecs…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsChaos-based Image/Signal Encryption · Generative Adversarial Networks and Image Synthesis
