ELANA: A Simple Energy and Latency Analyzer for LLMs

TL;DR

ELANA is a lightweight, open-source profiling tool designed to evaluate the latency and energy consumption of large language models across various hardware platforms, aiding optimization and research.

Contribution

We introduce ELANA, a simple, versatile profiler for analyzing LLM performance metrics on diverse hardware, supporting all Hugging Face models and energy logging.

Findings

Supports multi-GPU and edge GPU platforms.

Compatible with all Hugging Face models.

Includes energy consumption logging.

Abstract

The latency and power consumption of large language models (LLMs) are major constraints when serving them across a wide spectrum of hardware platforms, from mobile edge devices to cloud GPU clusters. Benchmarking is crucial for optimizing efficiency in both model deployment and next-generation model development. To address this need, we open-source a simple profiling tool, \textbf{ELANA}, for evaluating LLMs. ELANA is designed as a lightweight, academic-friendly profiler for analyzing model size, key-value (KV) cache size, prefilling latency (Time-to-first-token, TTFT), generation latency (Time-per-output-token, TPOT), and end-to-end latency (Time-to-last-token, TTLT) of LLMs on both multi-GPU and edge GPU platforms. It supports all publicly available models on Hugging Face and offers a simple command-line interface, along with optional energy consumption logging. Moreover, ELANA is fully compatible with popular Hugging Face APIs and can be easily customized or adapted to compressed or low bit-width models, making it ideal for research on efficient LLMs or for small-scale proof-of-concept studies. We release the ELANA profiling tool at: https://github.com/enyac-group/Elana.