Llama 2: Open Foundation and Fine-Tuned Chat Models

TL;DR

Llama 2 is a set of large language models, including chat-optimized versions, that outperform existing open-source models in benchmarks and safety, aiming to be a responsible open alternative.

Contribution

We introduce Llama 2, a new open-source LLM family with fine-tuned chat models, and detail our approach to safety and fine-tuning for community use.

Findings

Llama 2 models outperform open-source chat models on most benchmarks.

Llama 2-Chat models show high helpfulness and safety in evaluations.

Our approach facilitates responsible development of LLMs.

Abstract

In this work, we develop and release Llama 2, a collection of pretrained and fine-tuned large language models (LLMs) ranging in scale from 7 billion to 70 billion parameters. Our fine-tuned LLMs, called Llama 2-Chat, are optimized for dialogue use cases. Our models outperform open-source chat models on most benchmarks we tested, and based on our human evaluations for helpfulness and safety, may be a suitable substitute for closed-source models. We provide a detailed description of our approach to fine-tuning and safety improvements of Llama 2-Chat in order to enable the community to build on our work and contribute to the responsible development of LLMs.