Gemma: Open Models Based on Gemini Research and Technology

TL;DR

Gemma introduces lightweight open models based on Gemini research, demonstrating strong performance on language understanding, reasoning, and safety benchmarks, with a focus on responsible release and model development transparency.

Contribution

The paper presents Gemma, a new family of open models with 2B and 7B parameters, built from Gemini research, and evaluates their performance and safety comprehensively.

Findings

Gemma models outperform similar open models on 11 of 18 tasks.

Models demonstrate strong language understanding and reasoning capabilities.

Safety and responsibility evaluations are thoroughly conducted.

Abstract

This work introduces Gemma, a family of lightweight, state-of-the art open models built from the research and technology used to create Gemini models. Gemma models demonstrate strong performance across academic benchmarks for language understanding, reasoning, and safety. We release two sizes of models (2 billion and 7 billion parameters), and provide both pretrained and fine-tuned checkpoints. Gemma outperforms similarly sized open models on 11 out of 18 text-based tasks, and we present comprehensive evaluations of safety and responsibility aspects of the models, alongside a detailed description of model development. We believe the responsible release of LLMs is critical for improving the safety of frontier models, and for enabling the next wave of LLM innovations.