Accuracy and Consumption analysis from a compressed model by CompactifAI from Multiverse Computing

TL;DR

This paper evaluates CompactifAI, a compression method for Llama 3.1 8B, demonstrating significant resource reduction while maintaining accuracy, thus improving efficiency and scalability of large language models.

Contribution

It introduces and assesses CompactifAI, a novel compression technique that reduces energy consumption and computational resources without sacrificing model accuracy.

Findings

Compressed model significantly reduces energy consumption.

Model accuracy is maintained after compression.

Compression enhances scalability and cost-effectiveness.

Abstract

This study evaluates the performance of a compression method, called CompactifAI, developed by Multiverse Computing, applied to the large language model Llama 3.1 8B\cite{llama}. The evaluation focused on model efficiency (in terms of energy consumption) and accuracy using respectively the frameworks Codecarbon\cite{codecarbon} and Ragas\cite{ragas}. A comparison was performed between the model compressed with CompactifAI\cite{compactifai}\cite{compactifai2} and its full-size version. Our findings reveal that the compressed model using CompactifAI not only significantly reduced the computational resources but also maintained the model accuracy, making the model more efficient, scalable and cost-effective.