PocketLLM: Ultimate Compression of Large Language Models via Meta Networks

TL;DR

PocketLLM introduces a novel meta-network-based compression technique that significantly reduces the size of large language models by encoding weights into discrete latent vectors, enabling efficient storage and transmission with minimal accuracy loss.

Contribution

The paper presents a new compression method for LLMs using latent space encoding with meta-networks, outperforming traditional quantization and pruning methods at high compression ratios.

Findings

Compresses Llama 2-7B by 10x with negligible accuracy drop

Achieves superior performance at high compression ratios

Uses a simple encoder-decoder architecture with a codebook

Abstract

As Large Language Models (LLMs) continue to grow in size, storing and transmitting them on edge devices becomes increasingly challenging. Traditional methods like quantization and pruning struggle to achieve extreme compression of LLMs without sacrificing accuracy. In this paper, we introduce PocketLLM, a novel approach to compress LLMs in a latent space via meta-networks. A simple encoder network is proposed to project the weights of LLMs into discrete latent vectors, which are then represented using a compact codebook. A lightweight decoder network is employed to map the codebook's representative vectors back to the original weight space. This method allows for significant compression of the large weights in LLMs, consisting solely of a small decoder, a concise codebook, and an index. Extensive experiments show that PocketLLM achieves superior performance even at significantly high compression ratios, e.g., compressing Llama 2-7B by 10x with a negligible drop in accuracy.