Power-of-Two (PoT) Weights in Large Language Models (LLMs)

TL;DR

This paper explores Power-of-Two (PoT) weight quantization in Large Language Models to reduce memory and computational requirements, demonstrating promising results with minimal performance loss on GPT-2 models.

Contribution

It introduces PoT quantization for LLM weights, enabling efficient memory and computation reduction while maintaining model performance.

Findings

PoT quantization reduces memory usage significantly.

Computational efficiency improves via bit shifting instead of multiplication.

Performance degradation is minimal with 4-6 bits representation.

Abstract

Complexity of Neural Networks is increasing rapidly due to the massive increase in model parameters. Specifically, in Large Language Models (LLMs), the number of model parameters has grown exponentially in the past few years, for example, from 1.5 billion parameters in GPT2 to 175 billion in GPT3. This raises a significant challenge for implementation, especially for Edge devices where memory and processing power are very limited. In this work, we investigate reducing LLM complexity with special type of quantization, power of two (PoT), for linear layers weights and transformer tables. PoT not only provides memory reduction but more importantly provides significant computational reduction through converting multiplication to bit shifting. We obtained preliminary results of PoT quantization on Nano-GPT implementation using Shakespeare dataset. We then extended results to 124-M GPT-2 model. The PoT quantization results are shown to be very promising with cross entropy loss degradation $\approx$[1.3-0.88] with number of bits range [4-6] to represent power levels.